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During  the  Blue  Ribbon  Defense  Panel's  study  of  the  Department 
of  Defense,  its  research  staff  made  a  study  of  Operational  Testing  and 
Evaluation  (OT&E)  in  the  Department  of  Defense. 

The  composition  of  the  Panel's  OThE  Task  Group  was  as  follows: 

Dr.  Lawrence  G.  Starkey,  Staff  Consultant  (Task  Group  Leader) 
Mr.  Leslie  R.  Heselton,  Jr.,  Staff  Consultant 
Mr.  Morrill  E.  Marston,  Staff  Consultant 
Miss  Betty  J.  Neitsel,  Staff  Consultant 


•**  TW*! 


Individuals  Provided  by  the  Military  Services 

Colonel  Malcolm  J.  Agnew,  U.S.  Air  Force 
j  Mr.  Phil  E.  DePoy,  Department  of  the  Navy 

i  Lt.  Colonel  Benny  E.  Edney,  U.S.  Army 

Captain  John  J.  Hancotte,  Jr. ,  U.  S.  Navy 
!  Mr.  Herbert  W.  Hoyt,  Department  of  the  Air  Force 

Captain  Kenneth  F.  Rowell,  U.S.  Navy 
Colonel  Frank  L.  Taylor,  U.S.  Army 
i 

This  staff  report  to  the  Panel  is  considered  to  be  of  sufficient 
interest  to  top-management  personnel  of  the  Department  of  Defense  to 
be  included  as  an  Appendix  to  the  Panel's  Report.  However,  your 
attention  is  invited  to  Page  20  of  the  Panel’s  Report  which  states  that 
Staff  Reports  are  being  printed  as  information,  without  necessarily 
implying  endorsement  by  the  Panel  of  each  of  their  conclusions  and 
recommendations. 

Although  the  individuals  provided  by  the  Military  Services  per¬ 
formed  much  of  the  research  on  this  report,  it  is  emphasised  that 
they  are  not  responsible,  for  specific  findings,  conclusions,  or 
recommendations.  Additionally,  the  Task  Group  leader  felt  it  would 
not  bc'fair  either  to  these  individuals  or  to  the  study  to  attempt  to 
obtain  a  consensus  on  issues  that  are  often  controversial  and  on  which 
components  of  the  Department  of  Defense  may  and  frequently  do  differ. 
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I.  INTRODUCTION 


A.  Background 


The  Blue  Ribbon  Defense  Panel  selected  Operational  Testing 
and  Evaluation  (OT&E)  as  a  De gaVtaaCUtof- -Defense  activity  requiring 
study.  The  Panel's  decision  tliat  OT&E  merited  specific  attention 
was  based  on  a  large  number  of  judgments  to  that  effect  by  well 
qualified  groups  and  individuals  both  within  and  external  to  the  Depart¬ 
ment  of  Defense. 

For  example,  in  August  19f>9  the  Military  Aircraft  Panel  of  the 
President's  Scientific  Advisory  Committee  (PSAC1  recommended  to 
the  Secretary  of  Defense  that  an  OT&E  organization  be  established 
■*i  tne  Office  of  the  Secratary  of  Defense.  In  this  connection,  the 
PSAC  report  stated: 

"We  regard  the  creation  of  the  testing  and  evaluation 
group  as  of  the  utmost  importance,  since  we  believe 
most  of  our  previous  failures  to  be  prepared  for  wars 
we  fight  would  have  been  thoroughly  exposed  had  an 
adequate  program  of  testing  and  evaluation  existed. 

The  actual  tests  are  very  expensive  and  since  the 
Testing  and  Evaluation  budget  in  a  Service  is  often 
in  competition  with  funds  for  new  equipment  develop¬ 
ments,  we  believe  it  is  vital  that  the  Test  and  Evalua¬ 
tion  group  in  OSD  have  a  substantial  budget  to  allocate 
for  tests,  " 

At  about  the  same  time,  the  Defense  Science  Board  Task  Force 
on  RU)  Management  recommended  to  the  Director,  Defense  Research 
and  Engineering  that  ODDRbE  should: 

"beef  up  test  and  evaluation  functions  -  not 
only  on  weapon  system  products  but  also  with 
regard  to  policies  and  procedures." 
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In  the  Joint  Research  and  Development  Document  for  FY  1971- 
1988,  the  Joint  Chiefs  of  Staff  stated: 


"Test  and  evaluation  of  weapon  systems  and  materiel 
validates  achievement  of  performance  characteristics, 
exposes  weaknesses,  suggests  improvements,  and 
generates  requirements  for  new  programs.  It  plays 
an  essential  role  throughout  the  life  of  weapon  systems 
and  materiel.  Operational  test  and  evaluation  is  the 
final  essential  step  in  the  research,  development,  test 
and  evaluation  process;  as  such  :t  needs  increasing 
support  and  expansion  as  weapon  systems  become  more 
complex  and  interdependent," 

The  above  quotations  are  representative  of  a  large  body  of 
opinion  that  CTJ.E  m  the  Department  of  Defense  has  been  done  much 
less  well  than  it  should  have  been  and  that  there  is  a  potentially 
large  pay-off  for  performing  OT&E  more  effectively.  It  is  significant 
that  similar  concern  has  recently  been  expressed  by  such  agencies  as 
the  Bureau  of  the  Budget,  the  Comptroller  General  of  the  United  States 
and  Congressional  Committees  studying  the  acquisition  and  effectiveness 
of  DOD's  weapon  systems  and  materiel. 

It  must  be  emphasized  that  the  Task  Group  did  not  start  with 
the  assumption  that  OT&E  needed  extensive  overhauling  -  or,  for 
that  matter,  that  OT&E  was  currently  being  accomplished  in  a  large! 
unsatisfactory  manner.  The  Task  Group  did  accept  the  judgments  C  ,■  a 
above  as  ample  justification  for  the  study,  but  approached  the  study 
without  preconceptions  -  entirely  free  to  make  an  independent  assess 
ment  of  how  OT&E  is  currently  being  conducted. 

B.  Objectives 

The  principal  objectives  of  the  OT&E  study  have  been  to  assess 
the  effectiveness  of  the  organization  for  and  the  conduct  of  OTS.-E 
throughout  the  Department  o'  Defense,  to  determine  who  the  significant 
changes  are  required,  and  bused  -ipon  these  findings  to  de-elop  any 
required  policy  guidance  for  the  conduct  of  OTkE  of  weap-ns,  weapons 
systems,  and  supporting  systems  and  equipment. 
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C.  Method  of  Approach 

As  the  first  order  of  business,  the  Task  Group  developed  a 
methodology  for  conducting  its  study.  Tiiis  is  described  briefly 
as  follows: 


1,  Specific  Study  Tasks 

First,  it  was  essential  to  define  what  OTkE  actually  is  - 
that  is,  what  kinds  of  activities  it  encompasses  and  what  results 
it  may  reasonably  be  expected  to  produce.  This  was  not  easy, 
s:nce  it  was  necessary  to  define  it  in  terms  that  have  meaning  and 
relevance  throughout  the  Defense  establ'shment  and  not,  for  example, 
to  one  Service  alone. 

Next,  it  was  decided  that  it  would  be  useful  to  document 
selected  representative  OT&E  efforts  both  from  tfie  individual 
Military  Departments  and  joint  efforts  sponsored  by  OSD  or  JCS. 

The  case  histories  might  be  specific  operational  tests  or,  alternatively, 
development  programs  with  OT&.E  activities  and  their  influence  noted 
and  evaluated.  It  was  believed  that  preparing  such  case  histories 
would  yield  insight  into  the  characteristics  of  both  successful  and 
unsuccessful  OT&E. 


Concurrently,  it  was  judged  necessary  to  identify  and 
describe  existing  major  OT&-E  organizational  activities  throughout 
the  Department  of  Defense.  Through  this  means,  it  was  possible  to 
judge  the  importance  assigned  to  OT&E  and  to  understand  how  major 
DOD  elements  have  organ  _-d  to  perform  the  OT&E  function. 

It  has  been  often  sa:d,  and  the  Task  Group  agrees,  that 
individuals  with  experience  and  knowledge  of  OT^E  arc  in  very  short 
supply  throughout  the  Defense  establishment,  Oft.n  those  who  have 
such  capabilities  are  no  longer  associated  with  OT&E.  It  was  decided 
that  it  would  he  essential  to  consult  with  military  and  civilian  personnel 
with  recognized  expertise  in  OT&E.  More  than  60  such  individuals 
were  identified  and  engaged  in  in-depth  discussions  with  the  Task  Group. 


From  the  beginning  of  the  study,  it  was  clear  that  the 
Task  Group  must  assess  the  facilities  (ranges,  instrumentation, 

•tc.  !  available  tin uughout  the  OOD  tor  conducting  UU-b,  both  from 
the  viewpoint  of  current  adequacy  and  whether  they  can  fulfill 
requirements  for  future  OTfcE.  A  closely  related  subject,  which 
also  required  study,  is  the  adequacy  of  funds  for  the  support  of 
necessary  OTi-E  -  both  that  conducted  for  not  conducted)  by  the 
Services  and  the  joint  efforts  initiated  by  tiie  higher -than-Sorvicc 
levels  of  the  DOD. 

The  Task  Group  also  decided  to  investigate  the  extent  to 
which  large  private  industry  conducts  OTf. E,  in  order  to  evaluate 
whether  the  DOD  can  learn  worthwhile  lessons  from  that  source. 

2.  Critical  Issues 

The  Task  Group  concluded  that  the  objectives  of  the  OT h E 
study  could  be  most  .meaningfully  achieved  in  the  context  of  a  limited 
number  of  cr.tical  issues.  These  issues  were  expressed  as  questions 
lor  sets  of  related  questions)  for  which  answers  would  both  clarify 
the  current  DOD-wide  OThF.  situation  and  indicate  specific  actions 
which,  might  be  required  to  improve  that  situation.  These  important 
issues  are  as  follows: 

a.  What  is  encompassed  by  the  term  "operational 
testing  and  evaluation"?  What  are,  or  should  be,  the 
goals  of  OT&£? 

b.  A.t  what  points  in  the  life  cycle  c>f  a  military 
system,  and  in  what  ways,  can  OTl/E  be  useful?  Most 
pertinently,  can  OTwE  provide  meaningful  inputs  to 
military  decision-making7  If  so  -  where,  when,  and  in 
what  manner? 

c.  Arc  the  Military  Department*  accomplishing 

■  OT r-  E  to  the  extent  and  with  the  effecti  vcpc-hr  required? 
Wha»  organizational  relationships  witnin  the  Military 
Departments  most  effectively  support  product! /e  and 
objective  OTfcE  programs? 
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d.  Is  there  a  requirement  to  change  DOD  policy 

a  'd/or  organization  so  that  OTc.E  can  be  more  productively 
and  objectively  accomplished  and/or  the  results  of  OT&E 
move  effectively  used?  Specifically,  should  there  be  a 
’’higher-than-Service"  OT&E  organizatitn?  If  so,  where 
should  it  be  located  and  what  should  be  the  scope  of  its 
responsibilities  and  authority?  What  kinds  of  people 
should  be  assigned  to  such  an  organization?  ■  ■ 

e.  What  is  the  adeq  -acy  (capabilities  and  limitations) 
of  DOD  facilities  for  accomplishing  OTh-E?  Is  there 
unnecessary  duplication?  Is  there  a  need  for  new  and 
different  facilities  and/or  methods  for  controlling  their 
use? 

f.  How  should  OTkL  be  funded  throughout  the  DOD? 
Would  increases  in  funds  or  changes  in  methods  of  funding 
result  in  improved  OTkE? 
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II.  CONCLUSIONS 


1.  There  are  major  differences  among  elements  of  the 
Department-  of  Defense  as  to  what  Operational  Testing  and  Evaluation 
encompasses;  what  its  objectives  and  contributions  are,  or  should 
be;  and  what ‘organization  and  methods  are  requited  to  ?<  vor'ohnri 

it  most  effectively. 

2.  OTK.E  has  tour  principal  objectives: 

a.  To  provide  irformatior.  to  the  Research  &  Development 
community  that  will  assist  in  the  development  ot  in  w  Systems. 

b.  To  establish  the  characteristic?  and  capabilities  ant! 
limitations  of  such  systems  and  equipment  in  order  to  assist 
in  p'anning  force  size  and  force  structure. 

c. .  To  develop  tactics  and  techniques  for  employing  new 
systems  or  old  systems  which  must  be  adapted  for  new  uses. 

d.  To  determine  whether  weapons,  weapon  systems,  arid 
supporting  equipment  fulfill  up-to-date  military  requirements 
(which  may  be  substantially  different  from. the  requirements 
which  led  to  development). 

3.  OT&E  can  and  should  contribute  significantly  to  decision¬ 
making  at  all  levels  of  the  DOD.  However,  unless  the  process  of 
acquiring  military  materiel  is  radically  altered,  it  is  improbable 

.hat  OT&E  can  be  done  in  time  to  provide  "go-no  go"  recommendation! 
on  whether  to  commence  production  of  operationally -configured  major 
systems. 


4.  The  quality  of  OT&.-E  is  very  uneven.  There  is  :io  question 
that  military  OTbE  can  and  should  be  planned  and  executed  much  mo-c 
effectively  than  it  has  been  in  recent  years. 
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5.  The  results  of  the  OT&E  which  has  been  accomplished  have 
not  been  adequately  made  available  to  or  used  by  DOD  agencies  which 
need  them..  There  is  no  method  of  evaluating  and  preserving  such 
information. 

6.  OT&E  is  not  adequately  managed  or  supervised  at  OSD  level. 

7.  TheVe  is  a  requirement  for  an  OT&E  organization  at  "higher- 
than-Service"  level  to  insure  that  OT&E,  throughout  the  DOD,  is  more 
productively  accomplished  and  its  results  more  effectively  made 
available  and  used. 

8.  Existing  ranges  and  other  facilities  have  been  marginally 
adequate  to  support  the  OT&E  which  has  been  performed.  There  is 
well-founded  doubt  concerning  their  adequacy  for  OT&E  which  should 
have  been  bu  v'as  not  performed.  There  is  serious  concern  as  to 
whether  future  requirements  for  such  ranges  and  other  facilities 
can  or  will  be  met. 


|  9.  OTk-E  within  the  Services  is  done  most  effectively  when 

j  OT&E  organizations  report  directly  to  the  Chief  of  the  Service, 

representing  both  the  developer  and  user,  but  organizationally 
t  independent  ol  both.  There  are,  however,  considerable  forces 

J  within  the  Services  which  resist  the  independence  of  OT&E  organiza- 

.  tions. 

i  10.  There  has  been,  and  is  currently,  no  effective  means  for 

conducting  productive  joint  operational  tests  and  evaluations.  The 
!  fact  that  some  such  efforts  (for  example,  Joint  Task  force  Two) 

,  have  encountered  difficulties  and  achieved  few  useful  results  does 

not  obviate  the  requirement  for  much-needed  joint  OT&E. 

li.  There  should  be  a  specific  and  substantial  OT&E  budget 
(administered  by  the  "higher-than  Service"  OT&E  organization)  to 
allocate  for  the  most  needed  OTfcE  or  to  augment  Service  OT&E 
budgets  to  obtain  data  required  for  high-level  studies  cr  decision 
makers. 
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12.  There  is  a  shortage  of  experienced  and  capable  personnel 
directly  involved  with  OT&E.  There  has  been  inadequate  use  of 
civilian  scientists  and  operations  researchers/systems  analysts  in 
OT&F  at  all  levels,  but  particularly  where  operational  testing  i* 
actually  being  planned,  conducted,  and  analyzed. 

13.  Conduct  <>f  needed  OT &£  is  being  adversely  affected  by 
inadequate  funding  a..d  particularly  by  the  lack  of  budgetary  identity 
for  OT &E  fu  ids. 

14.  The  Weapons  Systems  Evaluation  Group  has  more  capability 
to  support  OT&E  than  is  being  used,  and  this  capability  could  be 
increased  by  juJiciour  assignment  of  military  personnel  and  the  use 

of  qualified  contractor  personnel. 
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III.  RECOMMENDATIONS 

1.  Establish  an  Operational  Testing  and  Evaluation  Group, 
with  civilian  leadership,  within  the  Office  of  the  Secretary  of  Defense, 
reporting  directly  to  the  Deputy  Secretary  of  Defense. 

2.  Consider  establishing  a  Defense  Test  Agency  with  broad 
authority  and  responsibility  for  DOD  test  activities  and  giving 
particular  emphasis  to  OT&E. 

3.  The  Secretary  of  Defense  should  communicate  to  the  Military 
Departments  the  importance  he  assigns  to  the  accomplishment  of 
productive,  objective,  and  timely  OT&E,  including  his  conviction 

that  the  cause  of  effective  OT&E  is  best  served  when  independent 
OT&E  organizations  report  directly  to  Chiefs  of  Service,  Service 
Secretaries  -  or  both. 

4.  An  early  task  and  continuing  responsibility  of  the  OSD  OT&E 
group  shcvld  be  to  develop  means  to  insure  that  productive  jc-int 
OT&E  is  accomplished  when  it  is  needed. 

5.  A  substantial  budget  for  OSD- sponsored  OT&E  should  be 
provided,  ^nd  administered  by  the  OT&E  group. 

6.  Require  the  Services  to  budget  separately  for  an  OT&E 
program  element. 

7.  Require  the  Weapons  Systems  Evaluation  Group  to  increase 
its  capability  to  perform  OT&E  task's.  Assign  selected  OT&E  tasks 
to  WSEG  as  it  develops  the  required  capability  to  accept  them. 
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IV.  DISCUSSION 


A.  The  Nature  of  Operational  Testing  and  Evaluation 

The  term  "operational  testing  and  evaluation"  is  not  well 
understood  throughout  the  DOD.  Probably  it  would  be  m-  "e  accurate 
to  say  that  it  means  different  things  to  different  people.  Understandably, 
there  have  been  many  attempts  to  define  OTfcE  with  rome  precision. 
These  efforts  are  generally  unproductive  -  or  counter-productive.  It 
is  not  desirable  to  force  OT&E  into  artificial  and  unnecessary 
constraints. 

A  major  goal  of  OT&E  is  to  determine  in  advance  the  effective¬ 
ness  (in  terms  of  capabilities  and  limitations)  ifi  the  ultimate  operational 
environments  (usually  combat)  of  military  weapons,  weapon  systems, 
supporting  systems  and  equipment,  tactics  and  techniques,  and 
organizational  arrangements.  It  should  be  emphasized  that  OT&E  is 
dynamic,  for  not  only  is  the  operational  environment  changing  but  so 
are  the  new  systems  and  the  new  uses  for  old  systems. 

It  has  been  customary  to  think  of  OT&E  in  terms  of  physical 
testing  (under  various  designations  such  as  operational  suitability 
testing,  employment  testing,  or  field  experimentation).  Operational 
testing  is  a  very  important  activity  (which  has  often  been  done  poorly), 
but  it  is  emphasized  that  the  goal  is  operational  evaluation  and  that 
physical  testing  is  only  one  means  of  performing  operational  evaluation. 
This  ia  at.  important  point  since  it  is  often  argued  that  operational 
testing  must  await  production  of  an  adequate  number  of  operationally- 
configured  systems;  and  by  this  time  ii  is  too  late  to  use  the  information 
gathered  to  help  decide  whether  to  produce  tire  new  system  or  even  to 
influence  in  any  significant  way  the  nature  of  the  system  procured. 

Thus,  OT&E,  as  a  total  process,  if  it  is  to  be  effective,  must 
extend  over  the  entire  life  cycle  of  a  system,  from  initial  requirement 
to  extending  its  life  by  adaptation  to  new  uses.  It  must  use  analytical 
studies,  operations  research,  systems  anal'/Bis,  "component  testing, 
testing  of  other  systems,  and  eventually  testing  of  the  system  itself. 
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Much  OT&E  docs,  however,  involve  physical  testing;  and 
therefore  it  is  important  to  distinguish  between  "functional”  testing 
and  "operational"  testing.  Unfortunately  this  distinction  is  often 
poorly  made  within  the  DOD  system  acquisition  process,  and  there 
is  a  disturbing  (and  apparently  increasing)  tendency  to  accept  the 
results  of  functional  testing  as  indicative  of  operational  capability. 

Functional  testing  (often  called  "engineering  testing")  is 
routinely  done  by  both  contractors  and  developing  agencies  of  the 
DOD.  It  is  intended  to  determine  how  well  various  systems  and 
materiel  meet  the  design  and  performance  specifications  which  were 
written  into  the  contracts  which  produced  them  -  in  other  words, 
whether  they  meet  technical  requirements. 

Operational  testing,  on  the  other  hand,  is  done  to  determine 
insofar  as  possible,  whether  such  systems  and  materiel  are  capable 
of  fulfilling  operational  requirements.  Operational  testing  must 
help  predict  in  advance  the  capabilities  and  limitations  of  systems 
when  they  are  subjected  to  the  stresses  of  the  environment  tor  which 
they  were  designed  (usually  combat).  It  must  take  into  consideration 
the  interface  with  other  systems  and  equipment,  tactics  and  techniques, 
organizational  arrangements,  and  the  human  skills  and  frailties  of  the 
eventual  users.  We  can  be  certain  that  knowledge  or  estimates  of 
the  operational  environment  will  change  significantly  during  the 
relatively  long  time  a  system  is  under  development,  and  operational 
testing  must  be  keyed  to  the  most  likely  conditions  of  use  rather  than 
the  requirements  for  which  the  system  was  developed.  A  system 
which  functions  perfectly  but  cannot  cope  with  the  threat  for  the  time 
period  when  it  is  available  is  worse  than  useless.  Operational  testing, 
along  with  continuous  operational  evaluation,  can  help  us  avoid  such 
unfortunate  situations. 


The  Role  of  OTbE  in  Military  Decision-Makir 


In  recent  years  there  has  been  a  growing  desire,  particularly  at 
the  OSD  level,  lo  use  data  from  OT&E  as  inputs  to  the  decision-making 
process.  The  decisions  that  have  to  be  made  at  this  level  concerning 
system  development  and  future  force  composition  are  often  very 
difficult.  Customarily  they  must  be  made  in  an  atmosphere  of  claims 
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and  counter  claims  by  proponents  of  competing  systems.  Often  there 
are  almost  no  hard  data  to  assist  in  the  decision  process.  Therefore,, 
especially  during  the  past  decade,  decisions  have  been  predominantly 
based  on  analytical  studies  -  which  have  in  turn  been  highly  sensitive 
to  important  initial  assumptions. 

Thus,  it  is  not  surprising  that  there  is  great  concern  about  the 
difficulty  3(  <  btainir.g  valid  quantitative  inputs  to  replace  or  support 
such  critical  assumptions.  Clearly,  it  would  be  very  valuable  to 
obtain  credible  quantitative  data  from  realistic  and  relevant  opera¬ 
tional  testing. 

Unfortunately,  it  has  been  nearly  impossible  to  obtain  test 
results  that  are  cither  directly  applicable  to  decisions  or  analyses  or 
which  can  be  credibly  extrapolated  for  that  purpose.  Often  test  data 
do  not  exist.  When  they  do,  their  usefulness  for  comparative  analyses 
is  generally  limited  seriously  by  the  fact  that  tests  arc  seldom  designed 
or  test  conditions  suffrciently  controlled  to  permit  valid  comparisons. 

It  is  especially  difficult  to  obtain  any  quantitative  information  in  time 
to  assist  in  decision-making. 

This  question  of  timeline-s  is  extremely  importarc.  For  this 
reason  it  is  essential  to  dispel  th »  v/idely-held  belief  that  useful 
OT&E  must  await  the  completed  product  of  R&D  -  that  it  is  or  should 
be  limited  to  the  testing  and  evaluation  of  production  systems.  It  is 
important,  as  will  be  indicated  below,  to  perform  OT&E  on  opera¬ 
tionally-configured  production  systems,  but  if  the  OT&E  process  only 
commences  at  that  point  it  misses  most  of  the  opportunity  to  influence 
that  product  on  behalf -of  the  operational  forces  -  the  ultimate  "users.  " 

There  are  four  important  ways  in  which  OT&E  can  provide  inputs 
to  m'litary  decision-making: 

1.  OT&E  can  provide  essential  inputs  to  help  with  decisions 
as  to  what  kinds  of  new  weapons  or  systems  should  be  developed 
and  what  capabilities  will  provide  worthwhile  increases  in  total 
force  effectiveness.  This  assistance  Is  in  the  form  of  obtaining 
and  making  available  accurate  quantitative  information  on  the 
capabilities  an*'  limitations  of  the  products  of  R&D  currently 
in  the  hands  of  -no  operational  forces.  Unless  it  has  such 


information  the  RiD  community  cannot  know  what  is  required 
to  ii:<-.*ease  capabilities  in  future  t:me  periods.  There  is  no 
doubt  that  such  quantitative  information  on  current  systems 
is  often  deficient  and  surrounded  with  uncertainties.  Such 
information  is  an  essential  element  in  validating  require¬ 
ments  -  that  is,  making  certain  that  requests  for  new  systems, 
and  the  risks  inherent  in  them,  arc  actually  justified. 

2.  OT&E  can  and  should  be  done  in  time  to  provide 
important  inputs  to  decisions  regarding  the  sire  and  composition 
of  military  forces.  Decisions  about  how  many  of  each  of 
several  systems  must  be  based  upon  some  degree  of  knowledge, 
or  estimates,  of  the  effectiveness  of  each  system  as  a  component 
of  the  force  mix  to  meet  a  spectrum  of  contingencies.  Realistic 
testing  can  provide  valuable  information  to  high-level  decision¬ 
makers  when  only  a  fraction  of  production  has  been  completed 
and  when  significant  alterations  to  overall  force  composition 
are  still  possible. 

3.  OThE  is  particularly  helpful  in  developing  tactics  and 
techniques  for  employing  new  systems  or  adapting  old  systems 
to  new  uses.  This  is  the  kind  of  OT&E  which  should  be,  and 
often  is,  conducted  by  the  operational  commands  equipped  with 
the  systems  in  question.  The  results  of  such  OT&E  are 
essential  to  the  operational  commander  in  making  decisions 
related  to  planning  fbr  and  conducting  combat  operations,  and 
in  devising  and  carrying  out  the  training  necessary  to  maintain 
operational  forces  in  a  state  of  readiness. 

4.  Traditionally,  an  important  objective  of  OTfcE  was  to 
test  production  systems  and  obtain  data  on  which  to  base 
decisions  as  to  whether  to  continue  or  discontinue  production. 
Unless  there  are  radical  changes  in  the  process  by  which  DOD 
acquires  major  systems,  it  is  unlikely  that  operational  testing 
can  play  the  major  role  in  such  decisions.  If,  however,  the 
practice  of  concurrency  were  lessened  and  true  prototypes 
were  built  for  pre-production  operational  testing,  then  OT&E 
could  contribute  in  a  major  way  to  such  "go-no  go"  decisions. 
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To  accomplish  the  important  purposes  cited  above,  an  effective 
program  of  OT&-E  must  start  with  the  earliest  requirements  and  continue, 
with  appropriately  changing  emphasis,  for  the  life  of  a  system.  It 
shouUI  be  pointed  out  that  to  conduct  this  "life-time  OT&E"  it  is  not 
necessary  to  conduct  a  different  type  of  testing  to  fulfill  each  purpose 
and  satisfy  each  "customer."  There  is  a  great  deal  of  overlapping, 
and  in  large  measure  a  generalized  program  of  OT&E  can  assist  with 
all  purposes. 

C.  OT&.-E  at  Higher-than-Service  Levels  of  POD* 

There  has  been  increasing  interest  in  OTkE  at  the  higher-than- 
Service  levels  of  the  DOD.  This  interest  is  mainly  attributable  to 
the  desire  to  obtain  hard  data  of  operational  validity  which  will  assist 
in  making  decisions  and  hopefully  reduce  the  chances  that  incorrect 
decisions  will  be  made. 

Despite  such  high-level  interest,  participation  in  OT&E  by  OSD 
and  JCS  has  been  limited  and  fragmented.  There  is  no  organization  at 
such  levels  with  the  overall  responsibility  for  either  deciding  what 
OT&E  should  be  done  and  for  what  purposes  or  for  following  through 
to  insure  that  the  desired  OT&E  is  done  in  a  scientific  and  timely 
manner  and  that  the  results  reach  those  who  need  them. 

The  OT&E  picture  at  OSD  and  JCS  is  dismal,  and  there  is  little 
if  any  indication  that  it  will  improve.  This  is  particularly  unfortunate, 
for  without  such  leadership  the  quality  and  extent  of  OTfcE  throughout 
the  DOD  cannot  help  but  suffer. 

1 .  OSD  QT&E  Activities 

The  Directorate  of  OTfrE  of  ODDR&tE  was  established  in  1966 
under  the  Deputy  Director  (Administration  and  Management).  1  Ithough 
establishment  of  this  organization  was  recognition  of  a  need  for  attention- 
to  the  operational  aspects  of  testing  and  evaluation,  its  authority  and 
resources  have  been  limited  from  the  beginning.  It  has  had  very  little 
influence  in  seeing  that  adequate  OTfcE  is  carried  out  in  the  DOD.  This 
small  organization  has  been  completely  military  and  has  lacked  both  the 
continuity  and  specialized  technical  expertise  necessary  to  formulate 
and  conduct  an  effective  program. 


See  Appendix  A  for  a  more  complete  description  and  discussion. 
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Furthermore,  the  location  of  the  organization  in  a  lower  echelon 
•r  of  a  developer  organization  has  militated  against  the  independence  of 

i  action  and  access  to  the  higher  levels  of  decision-making  which  are  t 

prerequisite  to  effective  representation  of  the  user. 

In  February  1968,  the  Assistant  Director  (Chemical  Technology), 

ODDR&E,  assumed  cognizance  over  the  Deseret  Test  Center  Joint 
~  Chemical/Biological  Operational  Test  &  Evaluation  activities  from  the 

(v  Assistant  Director  (OT&.-E),  ODDR&E.  This  was  a  further  dilution  of 

the  latter's  mission,  and  further  fragmentation  of  OTX-E  responsibilities 
within  OSD.  This  change  occurred  when  the  Army's  Dugway  Proving 
Ground  was  merged  with  the  Deseret  Test  Center.  Currently  the  part 
of  the  Deseret  Test  Center  program  involving  joint  offensive  weapon  • 

OT&E  faces  an  uncertain  future  because  of  the  Presidential  moratorium 
. .  in  this  activity. 

The  Assistant  Secretary  of  Defense  (Systems  Analysis!  has  been 
the  most  vocal  supporter  of  increased  OTtvE  for  and  by  OSD.  Until 
recently.  Systems  Analysis  prepared  Draft  Presidential  Memoranda 
on  General  Purpose,  Strategic,  and  other  military  forces.  Currently, 
it  performs  analyses  of  alternative  force  structures,  and  reviews 
Service  budget  requests  for  new  weapon  systems.  In  all  these 
endeavors,  Systems  Analysis  has  felt  the  need  of  OTf.E  data  not 
only  on  the  capabilities  of  new  systems  but  also  on  the  capabilities 
of  systems  in  the  hands  of  operational  forces. 

The  Weapons  Systems  Evaluation  Group  (WSEG)  was  established 
in  1948  with  the  mission,  among  other  thmg3,  of  conducting  evaluations 
for  the  Joint  Chiefs  of  Staff  and  the  Secretary  of  Defense,  WSEG  has 
been  reactive  rather  than  anticipatory  and  has  carried  out  OTf-E  through 
a  series  of  ad  hoc  studies  and  tests  which  were  organized  only  after  a 
major  problem  developed. 

Actually,  WSEG  has  only  been  minimally  concerned  with  actual 
involvement  with  the  design,  conduct,  and  analysis  of  physical  opera¬ 
tional  testing.  It  was,  however,  involved  in  the  1950's  with  the  large 
stale  joint  ECM  tests  and  exercises  called  WEXVAL.  More  recently 
it  had  a  major  part  in  establishing  Joint  Task  Force  Two  and  participated  , 

to  some  extent  in  its  activities  until  its  disestablishment.  In  1967-68, 
it  undertook  an  operational  test  of  the  M-16  rifle  in  a  simulated 
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environment  in  Panama;  this  OT&E  effort  is  generally  regarded  as 
highly  successful.  WSEG  receives  its  tasks  from  ODDR&E  and  JCS 
and  is  not  ordinarily  regarded  as  an  OT&E  organization. 

2.  JCS  OT&E  Activities 

At  the  present  time,  there  is  very  little  involvement  of  the 
Organization  of  the  Joint  Chiefs  of  Staff  in  OT&E,  and  apparently 
even  less  desire  to  be  so  involved.  The  JCS  do  not  consider  that 
OT&E  properly  falls  within  their  purview. 

The  JCS  have  from  time  to  time  sponsored, large  scale  joint 
exercises  and  joint  task  forces.  Some  of  the  latter  have  achieved 
considerable  permanence.  For  example,  Joint  Task  Force  Eight 
was  hurriedly  created  in  1961  to  facilitate  resumption  of  atmospheric 
nuclear  testing.  It  will  be  disestablished  in  July  1970. 

It  is  probable,  however,  that  much  of  JCS'  reluctance  to 
participate  directly  in  joint  OT&E  is  owing  to  Joint  Task  Force  Two, 
which  was  established  in  1964  and^ disestablished  in  1968.  JTF-2 
was  a  response  to  a  demonstrated  need  to  learn  more  about  the 
problems  associated  wnh  low  level  penetration  and  mission  performance 
by  aircraft.  There  is  •  rv  little  to  show  for  an  expenditure  on  the 
order  of  80  million  dollars;  however,  it  can  and  has  been  argued  that 
the  JCS  terminated  the  organization  just  as  it  was  on  the  verge  of 
producing  highly  useful  results. 

At  the  same  time  JTF-2  was  being  disestablished  (April  1968), 
the  then  Deputy  Secretary  of  Defense  (Mr,  Nitze)  requested  the  JCS 
to  consider  establishment  of  a  small  Joint  Test  and  Evaluation  agency, 
which  would  be  an  extension  of  the  Joint  Staff  similar  to  The  Special 
Assistant  for  Strategic'  Mobility.  The  JCS  lost  no  time  in  replying 
(May  1968)  that  they  had  considered  the  suggestion  and  had  concluded 
that  such  an  agency  was  unnecessary.  The  reply  went  on  to  express 
the  belief  that  there  already  existed  within  the  Organization  of  the  JCS, 
the  Services,  ana  other  agencies  the  capability  to  plan,  conduct,  and 
evaluate  the  results  of  operational  tests,  including  tests  which  involved 
joint  issues.  It  is  amply  evident  that  this  capability  does  not  exist  ard 
•  hat  the  ad  hoe  testing  on  which  the  JCS  relies  produces  very  little 
useful  data  to  support  high-level  decision  making. 
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The  JCS  might  have  argued  more  convincingly  that  there  was 
a  need  for  such  an  organization,  but  that  the  Organization  of  the  JCS 
was  not  the  place  to  establish  it. 

3.  Assessment  of  OT&E  at  Higher-than-Scrvice  Levels  of 
the  POD 

As  ind'cated  in  the  foregoing  discussion,  several  elements  of 
OSD  and  the  Organization  of  the  JCS  engage  in  some  OT&E  activity. 

The  principal  characteristic  of  such  activities,  however,  is  that 
they  are  fragmented.  None  of  them  has  the  responsibility  for  over¬ 
seeing  DOD  OT&£  as  a  whole  -  including  that  which  is  carried  on 
by  the  individual  Services.  In  fact,  such  higher-than-Service  OT&-E 
activities  appear  to  have  very  little  communication  with  each  other. 
There  is  no  focal  point  at  OSD  or  JCS  level  to  which  one  can  apply 
for  information  regarding  policies,  procedures,  organizations,  and 
facilities  for  conducting  OT&E  of  systems. and  equipment  within  the 
DOD. 

Thus,  there  is  no  organized  capability  at  higher-than-Service 
level  to  take  a  broader  view  toward  OT&E  than  is  possible  within  the 
Services,  The  requirements  for  data  from  OT&E  at  the  higher  levels 
differ  very  considerably  from  those  of  the  Services.  Some  mechanism 
is  needed  to  place  in  perspective  the  more  narrow  Service  viewpoints 
and  to  reduce  the  institutional  bias  which  is  preyalcnt  in  Service  OT&E 
undertakings.  No  effective  means  currently  exists  to  identify  to  the 
Services  the  OT&E  derived  data  which  are  required  at  OSD  and  JCS 
level  for  decisions  involving  development,  force  structure,  and 
contingency  planning. 

In  gono/fcl,  the  supervision  of  OT&E  of  strategic  missiles  by 
OSD  and  JCS  is  adequate.  The  Deputy  Director  (Strategic  and  Space 
Systems),  ODDR&E,  exerts  considerable  influence  on  OT&E  activity 
and  faciliti''H  for  evaluating  strategic  missile  performance,  in 
addition,  VVSEG  studies  in  the  area  of  OT&E  of  strategic  missiles 
have  been  effective  in  influencing  methods  for  such  evaluations. 

On  the  other  hand,  the  supervision  of  OT&E  of  non-nuclear 
systems  provides  the  best  example  of  fragmentation  and  ineffective 
higher -than -Service  supervision. 
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The  Assistant  Director  (OTJ.E),  ODDR&E..  has  the  principal 
responsibility  for  OSD's  interests  in  and  supervision  of  non-nuclear 
OTS.E  throughout  the  DOD.  This  directorate  has  b'-en  largely 
ineffective  for  the  four  years  it  has  existed.  In  part,  this  is  because 
of  inadequate  manning;  but,  more  importantly,  its  location  within 
ODDFt&E  (a  development  agency)  has  inhibited  effective  performance 
in  the  OT&E  of  the  products  of  the  development  process.  There  is 
extremely  .little  productive  relationship  between  the  Assistant  Director 
(OTk-E)  and  Service  OTfcE  personnel  and  agencies.  Sometimes  the 
latter  were  not  aware  that  there  was  such  an  organization  .  ;min  the 
OSD. 

Two  other  OSD  offices  monitor,  and  in  some  respects  supervise, 
aspects  of  non-nuclear  OTbE,  but  their  functions  duplicate  or  overlap 
those  a?signod  to  the  Assistant  Director  (OTbE).  These  offices  are 
the  Assistant  Director  (Chemical  Technology),  ODDR&E,  which  pro¬ 
vides  OSD  attention  to  joint  OT&E  of  chemical  and  biological  weapons, 
and  the  Defense  Communications  Planning  Group,  which  directs  both 
the  testing  and  operational  employment  of  certain  automated  sensor 
devices. 

JCS  relies  on  an  ad  hoc  approach  to  joint  OTbE.  This  technique 
which  usually  involves  establishing  a  joint  task  force  to  appiy  OT&.E  to 
important  problems  has  very  seri  »us  shortcomings.  First,  such 
problems  only  become  candidates  for  JCS  attention  when  they  become 
urgent;  JCS  has  no  mechanism  for  anticipating  such  problems.  Second, 
the  ad  hoc'  joint  task  force  approach  is  subject  to  delays  inconsistent 
with  a  sense  of  urgency,  and  the  problems  associated  with  mobilizing 
the  required  operational  and  scientific  expertise  arc  often  inadequately 
solved.  The  not-surprising  result  is  a  history  of  ineffective  joint  OTf.-E 
efforts. 

The  Weapons  Systems  Evaluation  Croup  is  responsive  to  the  JCS  for 
studio?  and,  upon  occasion,  OT&E.  For  example,  m  1967-1968,  WSEG 
condu<  ted  an  urgent,  effective,  ar.d  influential  operational  test  of  the 
M-  16  rifle.  WSEG  is  not  manned  either  with  officers  or  civilian  con¬ 
tractor  personnel  with  the  objective  of  maintaining  a  capability  for  OTk-E. 
Nevertheless,  it  is  evident  that  WSEG  has  a  greater  capability  in  the 
OTbE  area  than  is  being  used;  and  if  the  decision  were  made  to  do  so 
this  capability  could  be  substantially  increased. 
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D.  Operational  Testing  ?<  Evaluation  in  the  Services 

(r.  the  sections  describing  OSD  and  JCS  OT&E  activities,  there 
was  no  mention  of  any  control  or  important  influence  which  these 
higher  levels  have  on  the  OTf.-E  performed  by  the  individual  Services. 
Currently,  and  historically,  there  has  been  no  OSD  direction,  either 
f  irmal  or  informal,  in  this  area.  OT&E  done  by  the  Services  has  been 
and  remains  the  pierogati/e  of  the  Services. 

In  view  of  the  absence  of  any  regulation  or  guidance  from  higher 
authority,  it  is  not  surprising  that  the  Services  have  differed  substantial? 
both  in  OTJiE  philosophy  and  in  organization  to  carry  out  and  report  on 
OThE  acti  uties.  Methods  and  procedures  for  performing  OT&E  func*:  .-r 
have  changed  in  all  Services,  to  a  greater  or  lesser  degree,  over  the 
past  20-odd  years.  There  have  never,  however,  been  such  'arge 
differences  as  now  exist.  These  are  most  evident  in  the  matter  of 
organization.  There  are  three  basic  ways  to  organize  for  OTfcE: 

1.  An  independent  organization  reporting  directly  to  the 

Chief  of  Service, 

.2.  An  organization  under  the  developer. 

3.  An  organization  under  the  user. 

At  the  present  time,  all  of  the  above  organizational  alternatives  may  be 
found  in  the  Services. 

The  following  brief  account*  highlight  the  salient  features  of 
organization  for  and  conduct  of  JT&E  in  the  Services. 

1 .  OTfrE  in  the  Army* 

The  Army  system  of  testing  and  eva’uation  is  currently  being 
reorganized  to  place  more  emphasis  on  OT&E  -  and  particularly  on 


<Scc  Appendix  15  for  u  more  complete  description  of  Army  OT&E 
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acnievine  significant  operational  testing  as  early  ai  possible  in 
the  development  cycle.  The  objective  is  to  introduce  the  results 
of  valid  operational  tests  into  decisions  regarding  whether  to  produce 
a  system  -  and  the  extent  of  product* on.  This  means  a  continued  and 
increasing  over lapping  of  the  Engineering  Tests  which  determine 
whether  systems  and  equipment  meet  technical  specifications  and 
Service  (user-oriented)  Tests  which  provide  data  on  the  extent  to 
wh-ch  military  specifications  are  met.  The  principal  n.ethcd  of 
com  messing  time  and  testing  will  be  the  Operational  Service  Test 
which  is  scheduled  to  be  completed  and  evaluated  prior  to  a  decision 
to  commence  full  production.  The  objectives  of  these  revised  pro- 
cedutcs  are  unexceptionable;  however,  it  will  be  several  years  before 
•  t  can  be  determined  whether  they  l  ave  resulted  in  substantially- 
improved  OT&E. 

!r.  the  current  Army  system,  OT&E  is  subordinate  to  organiza¬ 
tions  which  are  also  responsible  for  development  of  the  systems  end 
equipment  tested.  Both  Engineering  Tests  and  Service  Tests  are 
performed  by  the  Army's  Test  and  Evaluation  Command  (TECOM) 
which  is  a  subordinate  element  of  the  Army  Materiel  Command  (AMC). 
TECOM  performs  Engineering  Tests  at  a  number  of  proving  grounds, 
and  Service  Tests  are  conducted  by  specialized  TECOM  boards  which 
represent  and  arc  collocated  with  elements  of  the  user,  TECOM  will 
also  be  responsible  for  performing  and  reporting  on  the  new  Opurat'onal 
Serv.cc  Tests.  However,  new  testing  regulations  will  require  formal 
written  comments  from  the  appropriate  Center  Commander  (Armor, 
Infantry,  etc.  )  to  lie  attached  to  reports  of  Operational  Service  Tests 
which  will  be  subrmttec,  directly  to  the  Department  of  the  Army, 

There  is  a  basic  problem  with  the  Army  system  of  OT&E  since 
the  developer  (albeit  through  a  subordinate  agency)  tests  and  evaluate* 
the  operational  nuitabi'ity  of  the  systems  developed.  The  Army's 
apparently  official  position  in  this  regard  is  that  despite  organizational 
affiliation,  TECOM,  by  virtue  of  individual  and  organizational  integrity, 
is  able  t<  maintain  independence  and  objectivity;  and  that  its  location 
in  the  Army  hierarchy  is  an  administrative  convenience  which  does  not 
adversely  affect  i>h  performance  of  OTfdi,  in  1%6,  and  again  in  1%^, 
the  organizational  location  of  the  Test  Boards  and  TECOM  was  reviewed 
at  high  level,  and  the  decision  was  to  maintain  the  status  quo.  Though 
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the  location  may  be  expedient,  it  is  far  from  ideal.  No  matter  what 
safeguards  are  built  into  the  system,  theoretically  it  is  not  in  the 
interests  of  unbiased  and  objective  OT&E  to  have  those  who  perform  ^ 
it  report  through  the  dereloper  to  the  Chief  of  Staff  level  where 
important  decisions  may  rely  extensively  on  test  results  or  expert 
but  basically  subjective  evaluations. 

For  the  past  decade,  the  Army  al  s  conducted  a  kind  of 
testing  which  it  has  designated  "field  experimentation."  Field 
experimentation  is  not  under  the  developer,  nor  is  it  under  the  user. 
Rather  it  is  performed  by  an  organization  which  attempts  to  represent 
the  inte-ests  of  both  developer  and  user:  the  Combat  Developments 
Command  Experimentation  Command  (CDCEC).  A  field  experiment 
is  described  in  Army  Regulation  71-3  as: 

"a  controlled  exercise  conducted  to  collect 
objective  data  on  £  specific  problem  area 
for  use  in  developing  or  evaluating  new  or 
improved  operational  and  organizational 
objectives,  concepts,  tactics,  techniques... 

Field  experiments  are  conducted  under 
controlled  and  instrumented  conditions. . .  " 

Field  experimentation  represents  a  forward  look  by  the  Arniy  in 
one  type  of  OT&E.  Results  obtained  by  field  experimentation  are 
quantitative  and  provide  useful  inputs  both  to  the  developer  and  the 
user.  It  has  also  had  a  salutary  effect  throughout  the  Army  in 
upgrading  testing  methodology  and  facilities.  There  is  considerable 
evidence  that  field  experimentation,  and  the  Army  personnel  who  have 
participated  in  field  experimentation,  have  achieved  an  increased 
awareness  in  the  Army  that  there  is  a  need  to  perform  OTfcE  more 
effectively  in  the  future  than  lias  been  done  in  the  past. 

There  in  also  little  doubt  that  field  experimentation  has  been  less 
successful  than  was  hoped  for  when  it  was  established  or  confidently 
anticipated  even  after  several  years  of  testing  operations.  It  appears 
to  have  become  in  recent  years  less  responsive  to  the  needs  of  the 
developers  and  ih"  operational  forces,  particularly  the  latter.  There 
is  a  growing  body  of  judgment  within  the  Army  and  elsewhere  that 
CDCEC' s  increasing  preoccupation  with  instrumentation  ar.d  simulation 
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(and  the  inevitable  loss  of  operational  realism)  is  a  disturbing  trend. 
There  appears  to  be  a  requirement  to  re-evaluate  both  the  objectives 
and  methods  of  field  exper. mentation  to  assess  the  extent  to  which 
this  innovative  and  conceptually  worthwhile  Army  testing  activity  is 
living  up  to  its  early  promise. 

2 .  QTkE  in  the  Navy  and  Marine  Corps- 

h«a_vv.  The  N'avy  system  cf  OT&E  has  two  principal  characteristics: 
(1)  it  is  mainly  implemented  by  an  independent  OTfcE  organization  which 
reports  directly  to  the  highest  Service  level  (Chief  of  Naval  Operations), 
and  (2)  there  is  a  formal  way  of  getting  operational  evaluation  (including 
some  relevant  operational  testing)  done  early  in  the  overall  testing  and 
evaluation  process. 

The  independent  Navy  organization  responsible  for  most  Navy  OT&E 
is  the  Operational  Test  and  Evaluation  Force  (OPTEVFOR).  OPTEVFOR 
reports  directly  to  the  Chief  of  Naval  Operations  on  the  results  of  its 
tests.  OPTEVFOR  is  a  relatively  small  organization  and  does  not 
command  forces  or  maintain  its  own  test  ranges,  instrumentation,  and 
other  facilities.  It  does  maintain  small  test  squadrons  and  detachments. 
The  total  strength  of  OPTEVFOR  is  approximately  1400  officers  and 
enlisted  men. 

The  emphasis  of  OPTEVFOR  is  on  conducting  OTS.E  without 
elaborate  measurements  or  sophisticated  test  design.  In  making 
evaluations,  there  is  great  reliance  on  the  judgment  of  individuals 
who  have  come  to  OPTEVFOR  with  recent  and  relevant  operational 
experience  and  \  n>>  after,  relatively  short  tours  (perhaps  two  years) 
will  return  to  operational  assignments.  Only  very  rarely  do  such 
individuals  serve  '.ong  tours  with  OPTEVFOR  or  return  for  subsequent 
tours. 


Technical  and/or  Operational  Test  and  Evaluation  is  presently 
conducted  on  all  weapon  systems  and/or  components  by  OPTEVFOR 
prior  to  release  for  procurement  except  where  Chief  of  Naval  Operations 
specifically  authoi.zes  limited  or  pilot  production  prior  to  completion 
of  testing,  Such  authorization  is  occasionally  considered  necessary  on 
complex  systems  such  as  aircraft  in  order  to  test  the  products  of 
production  type  tooling  and  to  facilitate  the  transition  from  prototype 
t )  production. 


-See  Appenr  ix  C  for  a  more  complete  description  of  Navy  and  Marine 
Corps  OT&E 
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The  Navy  stresses  operational  evaluations  at  an  early  stage 
of  development  -  that  is,  at  a  time  when  it  is  still  possible  to  make 
significant  changes  if  findings  warrant.  Obviously,  such  early 
evaluations  fall  far  short  of  what  is  often  believed  to  be  true  OT&E. 
However,  their  emphasis  is  on  the  early  assessment  of  operational 
suitability  and  timely  identification  of  p  tcntial  operational  shortcomings. 
It  is  believed  that  trading  off  operational  realism  fo’-  timeliness  makes 
a  worthwhile  contribution  to  placing  more  effective  systems  in  the 
hands  of  operational  forces. 

An  example  of  such  early  assessments  is  the  Naval  Preliminary 
Evaluation  (NPE)  which  is  conducted  on  naval  aircraft.  Very  soon 
after  first  flight  (within  90  days  or  so)  well-qualified  officers  will 
conduct  a  NPE.  This  will  involve  flying  the  first  or  second  aircraft, 
even  though  it  will  be  far  from  operationally  configured,  to  learn  all 
that  can  be 'learned  at  that  time  about  operational  suitability.  Usually 
there  will  be  subsequent  NPE's  of  the  aircraft  at  appropriate  intervals 
and  as  something  closer  to  an  operational  system  is  evolving.  The 
important  features  of  NPE's  ?.re  that  they  are  timely,  are  performed 
by  highly -capable  operational  personnel,  and  are  reported  to  a  high 
enough  level  so  that  necessary  actions  can  be  taken  promptly. 

Other  operationally-oriented  evaluations  occur  even  before  first 
flight.  For  example,  the  mock-up  of  a  proposed  cockpit  configuration 
may  be  evaluated  by  personnel  of  OPTEVFOR's  VX  squadrons  and  of 
the  Naval  Air  Test  Center  so  that  operational  inputs  on  behalf  of  the 
eventual  user  may  be  made  early  in  the  development  process. 

Before  Navy  aircraft  and  ships  are  approved  for  use  by  the 
operational  forces,  they  are  subjected  to  another  kind  of  operational 
evaluation,  which  is  much  closer  to  true  OT&E.  This  is  done  by  a 
Board  of  Inspection  and  Survey  (BIS),  which  is  composed  of  experienced 
officers  with  appropriate  Navy  backgrounds. 

When  an  aircraft  is  declared  ready  for  BIS  trials  by  a  NPE 
board,  a  board  consisting  of  especially-qualified  officers  from  the 
test  centers  involved  (for  example,  Naval  Air  Test  Center;  Naval 
Missile  Center  for  air-to-air  and  air-to-surfacc  missiles;  Naval 
Weapons  Evaluation  Facility  for  nuclear  weapons;'  etc.  )  is  convened 
to  conduct  the  tests  under  the  BIS  command  for  air. 
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When  ships  leave  the  shipyard,  they  are  given  an  acceptance 
trial  or  underway  trial  by  a  permanent  BIS  force  which  has  its  own 
officers  assigned,  some  direct  from  the  fleet,  others  with  specialized 
qualifications  in  various  ship  systems  and  shipyard  background.  Later 
before  the  contractor's  guarantee  expires,  usually  within  six  months, 
the  BIS  performs  a  final  contract  trial  on  the  ship  (or  submarine)  and 
makes  its  final  report  to  the  Secretary  of  the  Navy. 

BIS  trials  are  not  commenced  until  the  system  to  be  evaluated 
has  approached  operational  configuration.  The  BIS  reports  are  made 
directly  to  the  Secretary  cf  the  Navy  with  a  second  copy  also  sent  to 
the  Secretary  via  the  Chief  of  Naval  Material  and  Chief  of  Naval 
Operations  for  any  comments  they  may  have. 

The  Navy  is  probably  the  most  logically  organized  of  the  Services 
to  perform  objective  OT&£.  The  main  deficiency  in  Navy  OT&E  is  that 
it  generally' produces  fewer  hard  data  cn  new  or  existing  weapon  systems 
and  equipment  than  would  be  considered  desirable.  It  relies  too  much  on 
the  collective  judgment  of  well-quali.‘ied  officers,  and  does  not  take 
adequate  advantage  of  the  testing  techniques  that  are  available  for 
obtaining  measurements  of  scientific  validity.-  This  is  apparently 
Navy  OT&E  philosophy  and  not  any  inherent  inability  to  do  more 
scientific  testing  in  the  OT&E  phase.  It  does  mean,  however,  that 
higher  level  study  organizations,  such  as  Assistant  Secretary  of  Defense 
(Systems  Analysis),  find  it  very  difficult  to  obtain  scientifically  valid 
data  from  Navy  operational  tests  which  they  can  use  in  comparative 
analyses. 

The  Navy  OT&E  system,  however,  has  the  important  sd*- mintage 
of  direct  access  to  the  Chief  of  Naval  Operations  (and  upon  occasion, 
the  Secretary  of  the  Navy)  so  that  when  test  results  dictate  expedited 
action  there  are  no  intervening  echelons  with  built-in  delays  and 
interests  which  must  be  protected. 

Marine  Corps.  The  Marine  Corps  does  not  have  an  organization 
which  is  devoted  solely  to  OT&E.  Marine  Corps  OT&E  consists 
principally  of  expanded  service  tests,  troop  tests,  and  special  opera¬ 
tional  evaluations.  The  Commandant  of  the  Marine  Corps  tasks  the 
Commanding  General,  Marine  Corps  Development  and  Education 
Command  (MCDEC),  with  having  such  tests  done. 
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In  the  absence  of  a  dedicated  OT&E  organization,  the  Marine 
Corps  ordinarily  us~s  Fleet  Marine  Force  units  for  its  tests.  The 
MCDEC  develops  test  plans  in  cooperation  with  the  Fleet  Marine 
Force  units  involved,  and  when  the  tests  are  completed  MCDEC 
prepares  test  reports,  including  pertinent  recommendations,  and 
forwards  them  to  the  Commandant,  Marine  Corps. 

In  those  joint  programs  where  the  Army,  Air  Force  or  Navy  does 
the  development  and  testing,  the  Marine  Corps  participates  as  required 
and  provides  appropriate  resources  (funds,  personnel  and  equipment). 
When  another  Service  has  statutory  responsibility  for  developmental 
action  that  also  satisfies  a  Marine  Corps  requirement,  the  OT&E  done 
by  that  Service  is  usually  accepted  by  the  Marine  Corps  a?  the  basis 
for  proem ement.  For  major  items  of  equipment  (aircraft,  tanks, 
armored  amphibians,  weapons)  the  Marine  Corps  depends  on  the 
other  Services  for  the  technical  development,  and  in  most  cases  the 
operational  testing,  with  varying  degrees  of  participation  by  the 
Marine  Corps. 

The  Marine  Corps  Development  Center  located  at  Quantico, 
Virginia,  and  subordinate  to  MCDEC  is  the  principal  RDT&E  field 
activity  of  the  Marine  Corps.  It  also  maintains  a  West  Coast  Test 
Branch  at  Camp  Pendleton,  California.  Actually  the  facilities  at 
Quantico  have  limited  capability  for  OT&E  and  are  oriented  primarily 
towards  conduct  of  studies,  development  of  tactical  doctrine,  develop¬ 
ment  of  requirements,  supervision  of  Marine  Corps  sponsored  R&D 
programs,  and  monitoring  R&D  programs  of  other  Services. 

The  West  Coast  Test  Branch  is  mainly  concerned  with  service 
tests  on  equipment  peculiar  to  the  Marine  Corps  and  primarily  for 
employment  i  the  amphibious  environment.  It  docs  do  some  testing 
of  Army-developed  vehicles  to  determine  their  suitability  for  Marine 
Corps  use,  also  in  an  amphibious  environment. 

Requirements  for  funds  to  support  Marine  Corps  OT&E  are 
routinely  included  in  the  Department  of  Navy  RDT&E  appropriations. 
They  are  part  of  the  project  or  element  they  are  to  support.  There 
is  no  specific  program  element  for  Marine  Corps  OT&E. 


3.  OT&E  in  the  Air  Force*1 

The  Air  Force  currently  has  the  most  formal  and  structured 
system  of  testing  to  be  found  ir.  the  Services.  The  components  of  the 
Air  Force  testing  cycle  are  defined  with  considerable  precision  by 
Air  Force  Regulations.  Basically,  testing  and  evaluation  is  divided 
into  two  major  types:  Acquisition  Testing  and  Operational  Employment 
Testing. 

Acquisition  Testing  is  made  up  of  three  categories.  Categories 

I  and  II  are  essentially  RfcD  testing  and  are  the  responsibility  of  the 
Air  Force  Systems  Command  (AFSC).  Category  I  is  actually  performed 
by  contractors,  usually  using  contractor  facilities,  and  has  little  or 

no  operational  flavor.  In  most  instances,  Category  I  tests  are  of 
individual  components  and  subsystems.  Category  II  is  done  by  AFSC, 
witn  the  contractor  still  very  much  involved.  Ideally,  Category  II 
is  of  a  complete  system  in  as  near  an  operational  configuration  as 
practicable  at  that  stage  of  development.  In  actual  practice,  Category 

II  tests  seldom  are  operational  in  nature.  Both  the  operational  command 
which  is  to  be  the  ultimate  user  and  the  Air  Force  Logistics  Command 
(AFLC),  which  is  responsible  for  lifetime  support  of  the  system,  are  * 
required  to  monitor  Category  1  and  II  testing.- 

Category  III  is  the  first  Air  Force  testing  that  can  be  called  GTfcE. 
Category  III  comprises  tests  and  evaluations  of  operationally-configured 
systems  and  is  done  by  the  appropriate  operational  command  -  the 
ultimate  user.  Both  AFSC  and  AFLC  remain  involved  in  supporting 
roles. 

Operational  Employment  Testing  is  pure  OT&E.  It  is  conducted 
by  the  using  command  and  is  closely  related  to  integrating  the  new 
system  into  that  command.  Us  objecti  ves  include  the  development  of 
tactics  and  techniques  of  employment,  identification  of  operational 
problems  which  earlier  testing  may  not  have  revealed,  and  validation 
of  requirements  for  system  modification.  This  kind  of  testing  places 
great  emphasis  on  realism  of  environment  and  missions,  and  limits 
the  personnel  skills  and  support  to  those  that  would  be  available  in 
such  an  environment. 


w  See  Appendix  D  for  a  more  complete  description  of  Air  Force  OT&E 


I 


26 


Strategic  Ballistic  Missile  Testing  is  subject  to  unique  objectives 
and  constraints.  There  arc  Category  I  and  II  tests  which  are  roughly 
similar  to  those  described  above.  These  are  followed  by  Demonstration 
and  Shakedown  (DASO)  Tests,  which  arc  related  to  Category  III  testing. 
(DASO  is  prescribed  by  the  JCS  and  is  also  conducted  by  the  Navy  on  its 
strategic  missiles.  )  DASO  evaluations  are  conducted  by  the  operational 
command  (that  is,  the  Strategic  Air  Command),  assisted  by  the  agency 
having  Air  Force  engineering  responsibility.  Every  practicable  effort 
is  made  to  have  such  evaluation  performed  in  an  operational  environment 
using  operational  personnel  and  procedures.  The  degree  to  which  this 
is  achievable  is,  of  course,  less  than  for  systems  such  as  aircraft. 
Ballistic  Missile  Operational  Tests  come  closest  to  true  OT&E  of 
strategic  ballistic  missiles.  Their  main  objectives  are  to  assess 
operational  reliability  and  accuracy,  and  they  include  a  fixed  number 
of  launches  (though  not  from  operational  sites). 

There  arc  two  principal  problems  with  Air  Force  OT&E,  as 
currently  accomplished.  First,  operational  considerations  receive 
much  less  attention  than  desirable  during  Categories  I  and  II,  Second, 
the  operational  commands  responsible  for  Category  III  and  Operational 
Employment  Testing  lack  both  the  personnel  and  the  facilities  required 
to  do  a  first-rate  job  cf  OT&E. 

Air  Force  test  reports  must  proceed  through  several  echelons 
of  command  before  they  reach  Chief  of  Staff  level.  Furthermore,  there 
is  insufficient  management  and  supervision  from  that  level.  Relatively 
recently  (1965),  there  was  established  at  Air  Force  Headquarters  a 
Deputy  Director  of  Test  and  Evaluation  as  part  of  the  Directorate  of 
Operations,  Deputy  Ch>ef  of  Staff,  Plans  and  Operations.  This  OT&E 
oiganization  is  too  limited  in  its  authorized  scope  of  responsibility 
and  too  far  down  in  the  Headquarters  staff  to  influence  OT&E  to  the 
extent  necessary  to  achieve  a  high  degree  of  effectiveness.  The 
emphasis  which  is  placed  on  OT&E  also  varies  significantly  with  the 
backgrounds  and  predelections  of  successive  Directors  of  Operations. 

The  function  should  be  elevated  in  s'.ature  on  the  Air  Force  staff  if  it 
is  to  fulfill  its  very  worthwhile  objectives. 

There  are  valuable  lessons  to  be  learned  from  studying  the 
progress  of  OT&E  in  the  Air  Force  and  its  predecessor  elements  in 
the  Army,  the  Army  Air  Corps  and  the  Army  Air  Forces.  Historically, 
the  Air  Force  was  a  pioneer  in  OT&E.  From  its  origin  before  World 


War  II  until  its  disestablishment  in  1958,  the  independent  Air  Proving 
Ground  Command  (APGC)  at  Eglin  Air  Force  Base,  Florida,  reported 
directly  to  the  Chief  of  Staff  of  the  Air  Force  on  its  "operational 
suitability  tests."  For  years,  it  was  considered  a  major  Air  Force 
asset  and  is  generally  agreed  to  have  performed  well. 

In  retrospect,  it  appears  that  the  APGC  was  too  successful.  It 
grew  large  and  expensive.  With  its  growth,  it  became  more  cumbersome 
and  less  responsive  -  to  the  developers,  to  the  operational  commands, 
and  to  the  Chief  of  Staff  in  his  decision-makir.g  role.  When  one  of  the 
periodic  budget  squeezes  and  retrenchments  occurred  in  1957,  the 
APGC  was  vulnerable  and,  much  reduced  in  size,  was  relegated  to 
beirg  a  center  of  the  AFSC.  Subordinate  to  the  developer,  it  quickly 
lost  the  capability  to  do  effective  OT&E..  The  operational  commands 
were  unable  to  assume  OT&tE  responsibilities,  except  on  paper,  and 
there  is  little  doubt  that  some  of  the  Air  Force's  problems  when  it 
entered  Vietnam  can  be  traced  to  failure  to  do  certain  necessary  OT&E 
in  the  years  immediately  preceding. 

Over  the  years  since  1957,  there  have  been  attempts  to  re-establish 
the  APGC,  or  something  similar,  that  would  be  charged  with  performing 
independent  OT&E  and  reporting  results  to  Air  Force  Headquarters. 

The  consensus  within  the  Air  Force  now  seems  to  be  that  the  APGC  was 
indeed  highly  successful  at  one  ume  and  probably  could  be  again,  but 
that  it  would  be  a  luxury  the  Air  Force  could  not  afford  in  the  present 
austere  environment.  In  addition,  there  is  some  feeling  that  the  opera¬ 
tional  commands  arq  beginning  to  appreciate  the  value  of  comprehensive 
OT&E  and  that  present  programs  combined  with  increased  command 
emphasis  have  established  the  basis  for  significant  OT&E  improvement. 
Actually,  there  is  very  little  tangible  evidence  that  this  belief  is 
justified  and  considerable  evidence  that  the  operational  commands 
neither  understand  the  value  of  OT&E  nor  are  particularly  interested 
in  giving  it  a  high  priority  relative  to  other  command  responsibilities. 

There  is  a  lesson  to  be  learned  from  this  account,  which  apparently 
the  Navy  did  learn  with  regard  to  OPTEVFOR,  OPTEVFOR  has  never 
had  more  than  a  small  fraction  of  the  resources  that  the  APGC  had  and 
has  continued  to  operate  austerely.  The  APGC  was  a  well-manned, 
substantially  funded,  and  expensively  equipped  organization.  But  the 
APGC  no  longer  exists,  and  OPTE/FOR  has  survived  to  provide  the 
Navy  with  fhe  Services'  only  permanent  independent  OT&E  organization. 
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The  other  major  lessons  to  be  teamed  are  that  OT&E  does  not 
flourish  when  it  is  subordinate  to  the  developer,  and  that  assigning  the 
responsibility  for  OT&E  to  operational  commands  does  not  insure  that 
OT&E  will  be  done  effectively. 

4.  Assessment  of  OT&E  in  the  Services 


It  is  evident  that  the  Services  d:ffer  greatly  in  their  approach 
to  and  organisation  for  OT&E.  In  the  most  general  terms,  the  Navy 
stresses  independence  and  reporting  directly  to  the  Chief  of  Service. 

The  Army  has  generally  subord'rated  OT&E  to  the  developing  organiza¬ 
tion;  however,  its  emphasis  on  field  experimentation  represents  the 
most  innovative  ar.d  potentially  useful  form  of  OT&E  that  now  exists. 

The  Air  Force  has  now  placed  the  responsibility  for  OT&E  w:*.h  the 
operational  commands;  currently  these  commands  have  neither  th*.- 
kinds  of  personnel  or  facilities  required  to  do  a  good  job  of  OTArE. 

There  is,  of  course,  no  compelling  reason  why  the  Services 
should  have  parallel  OTt-E  organizations  or  should  approach  OT&H 
activities  with  the  same  philosophy  and  procedures.  There  is  also 
the  ancient  management  cliche  that  organizations  do  not  much  matter 
if  they  are  staffed  with  good  people.  Unfortunately,  in  this  instance,  . 
organization  does  seem  to  be  important;  and  in  an;  event  good  people  - 
that  is,  well-qualified  GT&E  people  -  are  tn  very  short  supply. 

OT&-E  in  the  Services  is  currently  in  transition  (perhaps  most  • 
so  in  the  Army).  Some  of  the  changes  have  undoubtedly  been  inspired 
by  the  desire  to  be  better  prepared  and  thus  avoid  some  of  the  difficulties 
encountered  in  Vietnam.  Some  arc  certainly  attributable  to  criticism 
from  higher  government  agencies  -  beth  in  and  out  of  the  Department 
of  Defense.  It  is  true  that  the  pressutes  of  Vietnam  have  inhibited  or 
rendered  hasty  and  ad  hoc  some  important  OTfcE.  This  appears 
particularly  true  of  the  Air  Force,  whose  OTbE  activities  are  in  some 
disarray  -  which  is  commonly  attributed  by  Air  Force  people  in  Vietnam. 

There  are  three  major  reasons  for  the  conclusion  that  Service 
OT&F.  has  been  of  uneven  quality  and  generally  much  less  successful 
than  would  be  desirable.  First,  OT&E  in  the  Services  has  lacked  much 
of  the  independence  that  encourages  objectivity  and  high  level  action 
when  ti.j  results  of  OT&E  call  for  it.  Second,  throughout  the  Services 
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there  ha*  been  very  little  guidance  from  high  levels  a*  to  what  is 
desired  from  OT&E  activities-  Third,  there  has  been  too  little 
support  and  encouragement  ol  OTfcE  from  high  levels  within  the 
Services.  This  means  that  not  only  is  more  funding  required  to 
support  OT&E  but  also  that  there  should  be  the  visible  indication 
of  how  OT&E  results  are  used  in  making  important  decisions. 

Insofar  as  can  be  determ.ned,  the  Service  Secretaries  have  had 
very  little  influence  on,  or  even  contact  with,  OT&E.  It  would  appear 
that  this  situation  should  change  and  that  the  Secretaries  should  take 
a  keen  interest  in  the  quality  of  Service  OT&E  and  the  potential  of 
OT&E  for  preventing  costly  and  embarrassing  failures  of  systems 
to  perform  in  accordance  with  requirements.  Currently,  Service 
Secretariats  are  inadequately  staffed  to  oversee  Service  OT&E 
activities;  however,  even  a.  minimal  effort  at  that  level  should  have 
very  beneficial  effects. 


Proposed  OT&E  Organization  in  OSD 


There  is  a  requirement  for  a  DOD  organization  at  higher-than- 
Service  level,  which  would  have  broad  responsibilities  and  authority 
for  OT&E  throughout  the  DOD.  It  is  evident  that  there  will  be  increased 
pressures  on  the  Secretary  of  Defense  to  assure  both  the  Congress  and 
agencies  of  the  Executive  Branch  that  programs  for  military  weapons, 
weapon  systems,  and  equipment  are  based  on  valid  operational  require¬ 
ments  and  that  they  receive  continuous  operational  evaluation  (including 
testing  when  possible  and  desirable)  throughout  the  process  which 
culminates  in  their  introduction  into  the  operational  forces.  Currently, 
the  Secretary  of  Defense  could  not  provide  such  assurance. 


There  are  a  variety  of  organizational  locations  for  such  a 
function  within  DOD,  However,  the  OT&E  function  needs  independence 
and  stature  if  it  is  to  perform  effectively. 


Consideration  was  given  to  assigning  the  OT&E  function  to  the 
Joint  Chiefs  of  Staff.  This  was  judged  an  undesirable  location.  The 
history  of  JCS-sponsorcd  test  organizations  has  not  been  encouraging. 
There  appears  to  be  little  desire  within  the  Organization  of  the  JCS  to 
assume  3uch  responsibilities,  and  the  JCS  arc  or.  record  as  opposing  the 
establishment  within  the  Organization  of  the  JCS  of  an  agency  sperifically 
devoted  tc  joint  OT&E. 
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It  appeared  most  logical  that  an  OT&E  function  should  be 
established  in  the  Office  of  the  Secretary  of  Defense.  The  OSD  badly 
needs  the  information  that  can  be  obtained  from  OT&E,  and  it  is  very 
desirable  that  an  organization  be  established  within  OSD  to  make  sure 
that  such  information  is  obtained  in  usable  form.  Unless  there  is  the 
capability  within  OSD  to  exercise  reasonable  and  appropriate  control 
over  DOD-w:de  OT&E,  it  will  be  necessary  to  rely  on  evaluations 
and  testing  designed,  performed,  and  interpreted  by  the  individual 
Services.  This  would  perpetuate  an  unsatisfactory  situation.  Although 
such  information  will  often  be  useful,  experience  has  demonstrated 
that  there  will  be  repeated  occasions  involving  conflicting  Service 
interests  when  it  would  be  preferable  for  OSD  to  specify  and  control 
the  scope  and  conditions  of  tests  and  evaluations.  At  the  very  least, 
OSD  should  have  detailed  knowledge  of  the  effects  of  the  conditions 
under  which  tests  are  conducted.  Moreover,  it  would  be  highly 
desirable  for  OSD  to  be  able  to  provide  guidance  to  individual  Services 
in  the  areas  of  OT&E,  based  cn  valid  requirements  for  a  continuing 
input  of  operational  test  data  suitable  to  support  comparative  analyses. 
Thus,  there  are  valid  and  significant  reasons  for  the  establishment 
of  a  comprehensive  OT&E  function  within  OSD,  . 

In  deciding  exactly  where,  within  OSD,  an  OT&E  group  should 
be  located,  several  basic  principles  served  as  guidance.  First,  it 
must  be  separate  from  the  developer;  this  rules  out  ODDR&E  as  a 
possible  location.  Second,  it  should  have  civilian  leadership  (albeit 
substantial  military  membership).  Third,  although  OASD  (Systems 
Analysis)  might  well  prove  to  be  the  main  user  of  the  data  produced, 
the  group  should  not  be  tied  to  one  "user"  organization.  Fourth,  an 
OT&E  group  should  have  a  separate  budget  for  OT&E,  Fifth,  the 
group  must  he  immediately  responsive  to  the  Secretary  of  Defense 
in  matters  relating  to  OT&E, 

In  view  of  all  these  considerations,  it  is  recommended  that  the 
organization  report  directly  to  the  Deputy  Secretary  of  Deferse,  If 
the  OSD  should  be  substantially  reorganized  in  the  future  to  include 
(as  has  been  suggested)  a  Deputy  Secretary  for  Evaluation,  the  OT&E 
group  would  properly  be  a  basic  part  of  that  element,  and  perhaps  be 
beaded  by  an  Assistant  Secretary. 

The  success  of  OT&E  at  OSD  level  will  be  highly  correlated 
with  independence,  support  by  the  Secretary  of  Defense,  effective 
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leadership,  and  the  ability,  experience,  and  dedication  of  the 
professionals  (civilian  and  military)  assigned  to  it. 

The  OSD  OT&E  group  would  have  one  principal  function  - 
that  is  to: 

--  Assume  responsibility  for  having  effective  OT&E 
done  within  the  DOD. 

Assumption  of  this  responsibility  would  be  dependent  upon  the  OT&E 
group  having  the  authority  required  to  perform  it.  This  authority 
would  have  to  include  control  over  tangible  assets  -  such  as  a  budget 
dedicated  to  OT&E. 

Other  specific  functions,  which  would  contribute  to  the  principal 
function  are  as  follows:  . 

--  Decide  what  the  critical  areas  for  OT&E  are. 

--  Determine  the  objectives  and  6cope  of  required  OT&E. 

,  --  Review  at  an  early  stage  of  R&D  the  plans  for  conducting 

OT&E  of  major  systems, ' 

--  Mo-;tor  the  conduct  of  OT&E. 

--  Insure  that  OT&E  results  are  disseminated  to  and  used 
by  agencies  an  1  individuals  that  need  them. 

--  Budget  for  OT&E. 

--  Promote  thr<  ’.ghout  DOD  the  coordination  and  exchange 
of  knowledge  and  idea  •  1  the  field  of  OT&E. 

--  Insure  that  joint  OT&r  is  done  to  the  extent  necessary. 

•  -  Represent  OSD  in  contacts  wi'h  external  agencies 
relative  to  OT&E. 
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Implicit  in  the  list  of  functions  above  is  that  the  OSD  CT&E  group 
will  not  direct  OT&E.  Rather  it  will  use  other  methods  of  insuring 
that  necessary  OT&E  is  done  by  the  organizations  where  much,  though 
probably  insufficient,  capability  currently  exists. 

It  is  not  anticipated  that  the  OSD  OTAE  group  would  be  large, 
but  it  would  be  expected  to  exert  influence  out  of  proportion  to  its 
size.  That  sire  might  fall  in  the  range  of  ?.G-40  professionals,  about 
equally  divided  between  civilian  and  military.  The  head  of  the  group 
should  be  civilian;  the  deputy  should  be  miliary.  The  civilian  component 
would  be  comprised  of  career  civ'l  servant'-,  and  scientists  on  relatively 
short  term  assignments  from  industry  and  .icademia.  The  stature  of 
the  group  should  be  such  that  it  can  attract  Sigh -caliber  civilian 
scientists'who  will  regard  such  an  assignment  as  an  opportunity  to. 
make  an  important  contribution  to  defense  effectiveness.  Hopefully, 
the  group  might  include  from  time-to-time,  and  particularly  in  ."'s 
early  stages,  individuals  who  have  been  calling  attention  to  the  lamen¬ 
table  lack  of  such  a  function  within  OSD. 

Military  members  should  come  from  all  ihe  Services.  It  is 
important  that  they  be  hand-picked  individuals  \  ho  can  be  productive 
from  the  beginning. 

The  OTbE  group  should  have  the  authority  and  funds  to  contract 
for  needed  research.  This  would  include  being  able  to  task  the  Weapons 
Systems  Evaluation  Qroup  for  research  which  that  organization  could  do 
well. 

The  specific  functions  of  the  OT&E  group  would  he  somewhat 
modified  if  a  decision  were  made  to  establish  a  Defense  Test  Agency 
(DTAl  which  would  have  broad  authority  and  responsibilities  for  all 
Defensp  tebt  and  evaluation.  A  DTA  would  be  concerned  with  the 
entire  DOD  test  program,  but  would  certainly  emphasize  operational 
testing,  particularly  mission-oriented  testing  which  cuts  across 
Service  lines  and  has  generally  been  poorly  done  in  the  past  -  or 
not  done  at  all. 
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Facilities  Available  for  Accomplishing  OT&E 


The  subject  of  OT&E  ranges  has  not  been  investigated  in 
depth  by  the  OT&E  Task  Group.  However,  discussions  with  key 
personnel  as  well  as  information  obtained  during  field  visits  to 
several  active  facilities  provided  seme  insights  into  the  major 
problems  and  issues  involved. 


The  necessary  characteristics  of  an  OT&E  range  are  just  as 
difficult  to  pin  down  as  is  a  precise  and  completely  acceptable 
definition  of  OT&E.  The  Services  have  few  facilities  dedicated  solely 
to  OT&E.  There  are  some  exceptions,  such  as  the  fairly  primitive 
capability  just  emerging  at  the  Tactical  Fighter  Weapons  Center, 

Nellis  Air  Force  Base.  Even  here,  however,  the  facility  is  also 
being  used  for  some  advanced  development  work  by  ARPA.  The 
scope  and  nature  of  OT&E  often  demand  instrumentation  as  sophis¬ 
ticated  and  precise  as  that  required  for  R&D  testing.  As  a  result, 
instrumented  OT&E  has  in  the  past  often  been  conducted  using  fixed 
R&D  facilities.  In  such  circumstances,  the  nature  of  this  type  of 
OT&E  is  influenced  by  the  capabilities  of  the  instrumentation  at  hand  - 
a  high  degree  of  precision,  but  with  little  flexibility  in  environment 
or  method  of  application  (tactics).  While  this  type  of  testing  can 
satisfy  many  OT&E  objectives,  the  broader  and  more  difficult  OT&E 
involves  the  appropriate  number  of  systems  used  as  the  operator 
would  during  combat  operations  -  in  short,  mission-type  tests.  For 
this  type  of  OT&E,  little  in  the  way  of  range  capability  exists.  While 
such  testing  is  ideal,  it  is  not  clear  how  expensive  it  would  be  (probably 
very  expensive)  or  what  results  could  be  sensibly  sorted  out  from  the 
many  complex  interactions.  In  this  respect,  the  Task  Group  has  noted 
the  proposed  HAVE  EDGE  project  of  the  Air  Force  for  an  integrated 
offensive  -  defensive  OT&E  test  capability  (all  service  requirements 
are  to  be  considered).  A  look  at  what  is  required  for  mission-oriented 
testing  can  shed  much  needed  light  on  this  subject.  At  this  time,  not 
enough  information  exists  to  satisfy  potential  service  users  that  such 
a  capability  will  be  practical  or  beneficial.  The  Air  Force  is  currently 
studying  this  matter,  with  contract  assistance. 


It  was  not  apparent  that  there  is  excessive  duplication  of  range 
facilities /capabilities.  Each  Servicc'3  capabilities,  as  developed, 
fulfill  legitimate  and  generally  unique  requirements.  Activities  v/ere 
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aware  of  others'  capacities  and  had  established  procedures  for 
application  of  use,  determining  priorities,  and  reimbursement  by 
other  users.  Exchange  of  facilities  information  seems  to  be  satis¬ 
factory  to  all  involved.  This  is  handled  on  an  informal  basis  by 
occasional  meetings  of  Range  Commanders  (Range  Commanders 
Council)  and  its  several  functional  committees,  such  as  the  Inter¬ 
departmental  Range  Instrumentation  Group  (1RIG).  No  one  at  any 
level  felt  this  type  of  inforrration  exchange  activity  would  benefit 
from  formalization  or  higher  level  supervision. 

It  appears  that  some  present  ranges  are  becoming  restricted 
in  capability  due  to  encroachment  on  airspace.  At  Eglin  Air  Force 
Base,  for  example,  there  is  very  little  opportunity  for  other  than 
prescribed  flight  routes  and  test  patterns.  At  the  Pacific  Missile 
Range,  Point  Mugu,  future  range  activities  and  planning  flexibility 
may  be  extensively  limited  by  oil  exploration  in  the  offshore  areas 
and  flight  patterns  of  the  proposed  Los  Angeles  Intercontinental 
Airport  near  Palmdale.  These  are  typical  problems  arising  from 
non-military  activities  that  can  interfere  seriously  with  DOD  range 
capabilities  and,  in  particular,  OT&E  types  of  testing. 

The  funding  of  ranges  and  facilities  appears  to  be  an  increasingly 
serious  problem.  At  the  present  time,  there  is  little  standardization 
in  methods  cf  funding  or  of  accounting.  Some  activities  prefer 
industrial  funding  on  the  basis  th^t  it  forces  planning  and  reveals 
inefficiencies.  Others  see  industrial  funding  as  leading  to  inflexibility 
and  eventual  loss  of  overall  DOD  service  and  capability,  and  insist  that 
some  "level-of-effort"  funding  is  necessary  to  insure  responsiveness. 
Accounting  methods  to  determine  reimbursement  costs  appear  to  vary. 
In  most  cases,  it  is  not  clear  that  direct  costs  of  using  ranges  are 
sufficiently  identifiable  from  overhead  costs  so  that  any  reasonable 
fiscal  basis  for  allocating  costs  is  available. 

The  Task  Group  considers  range  funding  and  cost  accounting  as 
a  problem  area  that  needs  high  level  attention  and  decisions.  This 
could  fruitfully  be  the  subject  of  subsequent  study  effort. 

No  activity  or  individual  at  any  level  could  see  benefit  in 
establishing  a  central  range  control  activity  viithin  OSD  or  as  a 
separate  agency.  The  existence  of  the  present  activity  in  ODDR&E 
(Assistant  Director  Ranges  and  Range, Support  under  the  Deputy 
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Director,  Strategic  and  Space  Systems)  was  known,  although  Service 
ranges  had  very  little  direct  contact  with  this  activity.  Its  functions 
were  not  well  known  at  the  working  level. 

It  is  the  feeling  of  the  Task  Group  that  a  presence  at  OSD  having 
cognizance  of  all  DOD  ranges  could  be  helpful  as  an  information 
coordination  activity  and  as  a  monitor /advocate  for  national  and 
Service  range  requirements.  In  this  way,  questions  of  unnecessary 
duplication  could  certainly  be  aired  and  resolved.  An  important 
function  of  such  a  presence  at  OSD  would  be  to  insure  that  Service 
requirements  are  exposed  to  higher  level  activities  that  appreciate 
the  magnitude  of  overall  range  resource  problems.  To  this  end,  the 
DOD  activity  would  function  as  a  liaison  to  activities  external  to  DOD 
that  are  involved  with  similar  activities  and/or  instrumentation 
(such  as  NASA,  Bureau  of  Standards,  etc. )  and  could  act  to  insure 
'.''at  the  Services  were  cognizant  of  the  activities  and  their  capabilities. 
In  addition,  the  OSD  function  would  insure  that  legitimate  defense 
OT&E  needs  are  given  proper  consideration  relative  to  private  and 
other  government  activity  pressures  regarding  use  of  land/air  space 
(such  as  Department  of  the  Interior,  FAA,  oil  companies,  etc. ).  In 
short,  a  DOD  level  activity  to  act  as  a  focal  point  for  range  matters, 
but  not  to  become  involved  with  scheduling  or  control,  is  considered 
necessary.  The  basis  for  this  is  the  recognition  that  the  overall 
resources  and  capabilities  involved  are  very  costly,  and  are  resources 
that  can  easily  become  lost  or  degraded  without  ,'n  i  level  visibility 
and  support. 

The  Task  Group  also  feels  that  OT&E  functions  at  OSD  level 
(addressed  elsewhere  in  this  report)  need  to  be  more  clearly  integrated 
with  range  resource  monitoring  activities.  The  current  organizational 
arrangement  with  range  and  OT&E  functions  under  different  ODDR&E 
Deputies  provides  too  much  opportunity  for  these  related  functions  to 
become  captive  of  diverse  and  contrary  directions  of  effort. 


Funding  of  OTkE  throughout  the  Depart  rient  of  Defense 


The  funding  of  OTf  E  throughout  the  DOD  is  a  major  unsolved 
problem.  There  is  general  agreement  among  people  engaged  in 
managing  and  performing  OT&E  that  funding  has  been  and  continues 
to  be  inadequate  to  support  much  necessary  OT&E. 
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Unquestionably  the  funding  of  OT&E  is  confused,  both  at  the 
OSD  level  and  within  the-  individual  Services,  a~.d  neither  in  OSD  or  in 
any  Service  is  there  a  single  agency  responsible  for  insuring  that 
OT&E  is  adequately  funded.  In  fact,  and  this  is  a  serious  indictment 
of  the  current  system,  there  is  no  such  agency  that  can  even  identify 
the  funds  that  are  currently  being  spent  on  OT&E. 

The  Air  Force  does  not  identify  OT&E  funding  requirements 
separately,  and  they  are  difficult  to  identify  in  the  Army  and  Navy 
budgets  because  funds  come  from  several  sources.  Because  funds 
"earmarked"  for  OT&E  do  not  have  separate  status  in  the  budget,  they 
are  often  vulnerable  to  reprogramming  actions  which  divert  them  to 
other  purposes.  When  it  is  time  to  perform  the  planned  OT&E,  it  is 
not  unusual  to  find  that  funds  have  disappeared  and  that  as  a  consequence 
OT&E  suffers. 

Fundi-g  of  OT&E  vuuin  the  individual  Services  differs  substantially. 
Some  examples  may  be  useful  in  illustrating  the  problems. 

Army 

The  Army  has  a  program  clement  for  RDT&E  which  supports 
TECOM's  six  specialized  Test  Boards  and  other  facilities,  and  pays 
for  most  of  the  personnel  who  perform  tests  and  provide  certain  test- 
related  services.  There  is  also  a  supporting  Military  Personnel 
Appropriation.  OT&E  funding  of  specific  projects  is  budgeted  for  by 
the  respective  Program  Managers.  Project  MASSTER  (based  at 
Fort  Hood,  Texas)  which  will  soon  be  doing  much  of  the  Army's  most 
important  OT&E  will  be  supported  by  separate  RDT&E  and  Operations 
and  Maintenance  (O&M)  funds.  Obviously,  Army  funding  of  OT&E  is 
fragmented.  Expenditures  for  OT&E  could  probably  be  identified  and 
compiled;  however,  such  data  do  not  now  exist. 

Navy 


The  Navy's  major  OT&E  activity,  OPTEVFOR,  is  supported  in  the 
budget  by  both  RDT&E  and  O&M  funds.  Tests  of  specific  systems  are 
budgeted  for  by  the  system  Program  Manager.  Once  again  there  is  no 
accurate  estimate  available  of  the  toial  cost  of  OT&E  in  the  Navy, 
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Air  Force 


OT&E  in  the  Air  Force  is  done  by  the  operational  commands  with 
some  management  guidance  from  the  Air  Staff.  However,  the  commands 
are  not  required  to  budget  specifically  for  OT&E,  and  such  testing  must 
be  paid  for  out  of  O&M  funds  -  that  is,  if  O&M  funds  arc  sufficient  for 
the  purpose.  There  is  nc  specific  provision  for  funding  OT&E  in  the 
Air  Force  FY  1971  budget  submission  nor  is  there  any  estimate  of  the 
funds  required  for  FY  1972-75.  No  current  Air  Force  Regulation 
specifies  who  should  budget  for  OT&E.  A  revised  Air  Force  Regulation 
55-31,  will  soon  require  better  planning  for  OT&E  funding,  but  there 
will  be  no  provision  for  protecting  or  even  accounting  for  such  funds 
once  they  are  included  in  overall  O&M  funds.  As  in  the  other  Services, 
OT&E3*^*  n°  aVailable  fJata  0n  what  5s  currently  being  spent  to  perform 

Clearly,  the  attention  given  to  OT&E  funding  throughout  the  POD 
is  not  consistent  with  the  growing  importance  assigned  to  that  activity 
at  high  levels  of  the  DOD  and  external  to  the  DOD.  There  have  been 
recommendations  within  the  Services,  within  OSD,  and  in  such  exter  lal 
agencies  as  the  Bureau  of  the  Budget  that  OT&E  should  be  a  separate 
program  element  in  the  budget.  There  is  resistance  to  this,  principally 
by  comptrollers  who  judge,  correctly,  that  this  would  reduce  their 
flexibility  to  reprogram  funds  to  meet  unexpected  contingencies. 

,  It  is  concluded  that  separate  program  elements  foc  OT&E  must  be 
established  within  the  Services  if  OT&E  is  to  receive  the  financial 
support  required.  Even  then,  OSD  must  assume  the  responsibility  for 
insuring  that  the  Services  budget  adequately  for  OT&E  There  is  con¬ 
siderable  evidence  that  the  Services  regard  this  function  as  less 
important  than  do  authorities  at  higher  levels  of  the  government. 

H.  Requirements  for  Joint  OT&E 

Most  of  the  OT&E  which  is  carried  on  within  the  DOI)  is  done  by 
individual  Services  and  Involves  the  systems,  equipment,  and  forces 
which  each  Service  lias,  or  would  like  to  have,  to  carry  out  assigned 
missions.  Currently  there  is  no  effective  method  for  conducting  OT&E 
which  cuts  across  Service  linos.  This  is  particularly  unfortunate  since 
in  most  actual  combat  environments  the  Services  must  conduct  combined 
operations.  The  interactions  between  Services  become  extremely 
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important  during  combat,  and  critical  military  missions  transcend 
Service  boundaries  and  responsibilities  (for  example.  Close  Air  Support, 
Reconnaissance,  Air  Supply).  Because  of  the  lack  of  joint  OT&E,  it  is 
not  only  very  difficult  to  predict  combat  capability  in  advance  but  it  is 
also  difficult  to  make  decisions  relating  to  overall  DOD  force  composition. 

The  Services  nearly  always  resist  evaluations  of  joint  capabilities. 
The  reason  for  such  resistance  is  clear:  such  evaluations  inevitably 
involve  the  roles  and  missions  of  individual  Services,  and  these  roles 
and  missions  have  never  beer,  clearly  defined  (or  at  least  are  interpreted 
differently).  Consequently,  these  unresolved  differences  constitute  a 
formidable,  and  sometimes  impenetrable,  barrier  to  the  conduct  of 
effective  joint  tests  and  evaluations. 

The  history  of  joint  OThE  in  recent  years  presents  a  dreary 
picture.  The  large  joint  tests  and  exercises  which  have  been  conducted 
seem  to  have  generated  a  maximum  of  disagreemert  (including  genuine 
ill  f.'-ling)  and  a  minimum  of  useful  information.  Two  examples  are  cited 
very  briefly  here;  it  is  believed  that  they  are  representative  of  the  problems 
encountered. 

1.  Tests  of  Army  Air  Mobility. 

In  1961-62,  at  the  direction  of  the  Secretary  of  Defense,  the 
Army  organized  a  study  effort  designed  to  explore  ways  of  increasing 
Army  air  mobility.  In  support  of  this  effort,  the  Army  conducted  a 
series  of  unilateral  tests.  The  Air  Force  was  quick  to  express  concern 
about  the  Army's  activities  and  convened  a  board  of  general  officers 
and  a  support  staff  to  study  related  capabilities  and  to  evaluate  the 
report  of  the  Army  to  the  Secretary  of  Defense.  The  Air  Force  board 
also  had  unilateral  testing  done. 

It  was  painfully  clear  that  from  the  Air  Force  point  of  view  the 
Army's  study  and  testing  represented  a  threat  to  assigned  Air  Force 
roles  and  missions  -  particularly  in  the  areas  of  Close. Air  Support 
and  Air  Transport.  It  was  equally  clear  that  the  Army  was  indeed 
using  the  opportunity  to  attempt  to  wrest  from  the  Air  Force  certain 
support  functions  to  which  it  believed  the  Air  Force  was  giving 
inadequate  attention. 


It  should  not  be  surprising  that  the  results  of  the  unilateral 
Army  and  Air  Force  testing  were  often  markedly  different,  especially 
when  the  testing  involved  activities  where  assigned  roles  and  missions 
appeared  to  overlap.  The  Air  Force  report  to  the  Secretary  of  Defense 
took  specific  exception  to  the  relevance  and  validity  of  the  Army  testing 
which  was  cited  in  the  Army  report. 

The  Army  and  Air  Force  were  directed  to  participate  in  joint 
testing  of  air  mobility  concepts.  Testing  was  directed  by  the  U.  S. 
Strike  Command  and  consisted  of  two  major  joint  exe-cises,  GOLD 
FIRE  I  and  GOLD  FIRE  II.  These  exercises  involved  brigade  and 
division  size  Army  units  and  appropriate  Air  rorce  supporting  units. 
They  produced  very  little  quantitative  information;  and  in  the  absence 
of  any  way  to  obtain  quantitative  measurements  of  performance, 
reliance  was  placed  in  the  judgments  of  experienced  officers.  Not 
surprisingly,  there  was  remarkably  little  agreement  between  the 
Services. 


2.  Joint  Task  Force  Two 

The  second  example  of  joint  testing  and  evaluation  involves  Joint 
Task  Force  Two,  which  was  created  in  1964  and  disestablished  in  1968. 
The  purpose  of  JTF-2  was  to  fill  troublesome  gaps  in  basi  :  knowledge 
about  low-altitude  penetration  and  operations  of  tactical  and  strategic 
aircraft.  The  lack  of  this  information  had  handicapped  both  the  R&D 
community  and  high  level  decision-makers  responsible  for  force  com¬ 
position  and  operational  contingency  plans. 

JTF-2  involved  a  minimum  of  inter-Service  rivalry;  for  the  most 
part,  roles  and  missions  were  not  believed  to  be  in  jeopardy.  Neverthe¬ 
less,  the  testing  activities  of  JTF-2  encountered  increasing  resistance 
from  all  the  Services,  which  was  undoubtedly  intensified  by  the  demands 
of  Vietnam  air  operations.  The  Services  were  required  to  support  JTF-2 
with  resources  of  personnel  and  aircraft  at  a  time  when  they  could  ill 
afford  either.  It  is  also  fair  to  say  that  the  Services  could  not  in  general 
ace  how  the  testing  done  by  JTF-2  was  benefiting  them.  Furthermore, 

‘.he  Organization  of  the  JCS  did  not  regard  it6elf  as  a  user  of  the  infor¬ 
mation  produced  by  JTF-2.  As  a  result,  JTF-2  was  almost  from  the 
beginning  on  the  defensive,  and  eventually  perished  because  it  lacked 
a  sufficiently  powerful  sponsor.  OASD  (Systems  Analysis)  and  ODDR&E 
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should  have  been  powerful  advocates  of  JTF-Z.  In  fact,  they  did  not 
support  it  when  it  needed  supporting  and  acquiesced  in  its  demise 
(while  unofficially,  '‘off-the-record,”  regretting  it). 

In  retrospect,  it  appears  that  for  most  of  its  existence  JTF-2 
was  in  an  organizing  and  learning  phase.  It  is  certain  that  it  was 
handicapped  by  the  constant  necessity  to  do  battle  for  resources  and 
to  justify  its  existence.  The  problems  it  faced  were  difficult,  mistakes 
were  inevitably  made,  but  much  was  learned  about  organizing  to  conduct 
complex  operational  tests.  It  is  probable  that  JTF-2  was  disestablished 
at  a  point  when  it  was  about  to  produce  information  that  was  needed  and 
that  we  still  need  and  do  not  have  any  way  of  obtaining.  When  disestab¬ 
lished,  it  was  probably  also  approaching  the  capability  of  taking  on 
other  joint  testing  which  very  badly  needs  doing.  It  is  particularly 
unfortunate  for  the  future  cf  OT&E  that  JTF-2  was  abandoned  when 
much-needed  instrumentation  and  analytical  tools  were  only  partly 
developed.  For  the  most  part,  this  investment  in  the  future  vas  lost. 

It  is  arguable  that  the  dissolution  of  a  capability  which  had  cost  on  the 
order  of  80  million  dollars  was  very  short-sighted.  . 

3.  Lessons  Learned  from  Past  Efforts  to  Conduct  Joint  OT&E. 

The  principal  lessons  to  be  learned  from  the  generally  unproductive 
efforts  to  conduct  joint  OT&E,  of  which  the  above  are  examples,  are  the 
following: 

a.  They  will  be  resisted  by  the  Services  whenever  they 
involve  roles  and  missions  -  and  the  joint  OT&E  with  the 
greatest  potential  will  involve  roles  and  missions. 

b.  The  Services  are  unsympathetic  to  expending  resour.es 
for  basic  information  which  they  cannot  see  will  help  them  in 
the  short  term. 

c.  If  joint  OT&E  is  to  be  productive,  it  must  be  directed 

by  a  civilian  sponsor  at  OSD  level  (not  JCSI),  sufficiently  powerful 
to  insure  that  the  objectives  of  the  OT&E  are  realized. 

Currently,  we  are  left  with  ad  hoc  arrangements  as  the  only  way 
of  organizing  for  and  conducting  joint  OTf<E.  This  has  assured  that  such 
joint  OTfcE  as  is  done  will  bo  largely  unproductive  and  of  little  use  to 
decision-makers  at  OSD  level. 
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4.  Value  of  Large-Scale  Exercises  for  Joint  Evaluations. 


There  is  a  lung  histot  ■;  of  large-scale  exercises  and  maneuvers 
more  or  less  routinely  conducted  by  elements  of  the  DOD.  Sometimes 
they  have  involved  only  one  Service,  but  more  often  two  or  more  Services 
have  participated.  Although  there  have  been  very  few  such  undertakings 
during  the  past  several  years  owing  to  the  demands  of  Vietnam,  it  is 
highly  probable  that  they  will  be  conducted  again  in  the  future  as  resources 
become  available. 

It  wou’d  be  very  desirable  to  use  these  large-scale  exercises  and 
maneuvers  \oth  to  assist  with  join-  evaluations  and  to  provide  basic 
data  on  joint  operational  capabilities.  They  provide  an  excellent 
opportunity  to  evaluate  systems  and  tactics  in  the  nearest  peacetime 
approximation  to  a  realistic  combat  environment.  Frequently  problems 
(for  example,  electronic  interference  and  conflicts  in  communications 
requirements)  which  do  not  occur,  or  at  least  are  not  noticed,  in 
individual  system  testing  are  identified  in  such  exorcises.  In  addition, 
when  it  is  possible  to  derive  assessments  of  system  capabilities  from 
large-scale  exercises,  they  are  generally  closer  to  actual  coml.at 
performance  than  are  estimates  based  on  individual  system  evaluation. 

In  the  past,  such  major  exercises  have  produced  very  little  of 
what  might  be  termed  "hard  data-'  -  that  is,  valid,  quantitative  informa¬ 
tion  which  could  be  used  to  support  objective  evaluations  of  capabilities.  ' 
On  the  other  hand,  most  such  exercises  have  produced  qualitative 
judgments  by  observers  selected  for  their  relevant  experience.  Thus, 
evaluations  of  large  joint  exercises  customarily  are  afflicted  by  the 
same  problems  described  in  the  earlier  example  of  joint  Army-Air 
Force  testing.  It  is  certain  that  joint  issues  can  rarely  if  ever  be 
resolved  by  military  judgment.  Thus,  there  is  a  great  premium  on 
any  methods  lor  obtaining  quantitative  data  which  are  subject  to 
scientific  verification. 

Unfortunately  there  is  a  natural  conflict  between  evaluation, 
and  its  inherent  data  and  information  collection,  and  the  principal 
purposes  of  most  large-scale  exercises*  training,  system  integration, 
and  identification  of  problem  areas.  Data  collection  often  interferes 
with  normal  operations  and  reduces  the  degree  of  realism,  and  since 
data  collection  is  normally  a  low-priority  objective,  attempts  to  obtain 
information  suitable  to  support  evaluations  often  meet  with  little  success. 
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The-e  is  ar.  urgent  need  to  exploit  such  large-scale  exercises 
as  sources  of  q»-antita».ve  information.  It  will  require  high-level 
support  to  establish  the  obtaining  o£  quantitative  data  as  a  hig*’- priority 
objective  of  at  least  some  exercises  -  or  some  phases  of  exe*  cises. 
Operations  a~d  systems  analysts  must  beccme  involved  in  the  design 
and  conduct  of  exercises  so  they  can  devise  methods  of  obtaining 
certain  much-needed  quantitative  information  with  the  least  possible 
degradation  to  operational  realism  ar.d  to  {be  time-honored  and 
important  goals  of  such  exercises. 

It  is  anticipated  that  an  OT&.-E  group  in  the  OSD  would  take  the 
initiative  in  making  such  large  ano  costly  exercises  produce  data  which 
would  be  very  useful  at  that  level. 

I.  OT*.E  in  Industry 


The  OT*<E  Task  Group  investigated  similarities  and  differences 
between  the  way  OTkE  is  done  in  the  DOD  and  in  some  major  U.  S. 
industries.  The  automotive  indurtry  was  represented  by  the  General 
Motors  Corporation  and  the  airlines  by  Pan  American  and  -.merican. 

The  organization  of  General  Motors  is  in  some  important  ways 
remarkably  analogous  to  tiie  DOD.  The  individual  aulomo  ive  divisions 
can  be  compared  with  the  military  departments,  and  the  corpoiate  bead- 
quarters  performs  functions  similar  to  the  OSD.  Each  auiw motive 
division  conducts  its  own  research,  most  often  internally  but  at  times 
using  facilities  provided  by  the  corporation.  The  corporation  provides 
overall  policy  guidance  and  itself  performs  research  in  areas  of  con¬ 
cern  to  several  divisions,  o (  current  interest  to  no  division,  or  so 
important  that  corporate  interest  is  required. 

The  major  difference  between  the  automotive  industry  and  the  DOD 
is  that  the  former  is  a  producer,  interested  primarily  in  profit,  while 
the  latter  is  a  user,  interested  primarily  in  performance.  This  different 
orientation  bears  heavily  on  the  amount  of  risk  judged  acceptable.  Auto¬ 
motive  research  and  testing  is  directed  mainly  at  product  improvement, 
with  the  assumption  that  competitors  are  constrained  to  act  in  accordance 
with  the  same  general  philosophy.  Military  research  and  testing  is  directed 
toward  attaining  large  increases  in  operational  capabilities  which  often 
challenge  the  state-of-the-art. 
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Most  OTfc.E  in  the  automotive  industry  is  in  connection  with 
production  rather  than  research.  There  is  considerable  emphasis, 
apparently  increasing,  on  OT&E  of  total  automotive  systems  to 
determine  which  components  require  improvement.  Research 
testing  (mostly  non-operational  in  nature)  is  generally  performed 
on  individual  components  rather  than  cn  the  complete  system. 

Only  a  limited  amount  of  system  testing  is  conducted  prior  to 
entering  actual  production  since  an  automobile  is  essentially  a  com¬ 
bination  of  components  with  veil  understood  characteristics. 

Pre-production  testing  is  done  to  check  the  assembly  and  insure 
that  no  mistakes  were  made  in  the  engineering.  Most  of  the  actual 
operational  testing  and  evaluation  is  conducted  on  new  production, 
primarily  for  quality  control  or  product  assurance.  In  fact,  all  of 
the  General  Motors  proving  grounds  are  administered  by  thi  Product 
Assurance  Division.  Each  new  model  is  tested  extensively  by  both 
the  producing  division  and  the  corporate  staff  to  insure  that  it  meets 
the  specified  requirements.  These  tests  are  conducted  in  conditions 
which  approximate  normal  usage  to  the  extent  possible,  and  use  many 
average  drivers  in  addition  to  trained  test  drivers.  In  addition  to 
product  assurance,  these  extensive  tests  help  to  identify  areas  of 
potential  product  improvement  for  future  models  in  order  to  increase 
customer  acceptance  of  the  product. 

Recent  government  regulations  are  beginning  to  force  the 
automotive'  industry  farther  into  fields  of  new  development  which 
will  involve  very  extensive  testing.  Various  safety  standards  have 
led  to  testing  and  evaluation  designed  to  determine  the  best  and  most 
economical  methods  for  compliance.  rhe*  have  also  led  to  increased 
operational  testing  of  the  effects  of  collisions  in  order  to  determine 
overall  vehicle  safety.  Emphasis  on  ant, -pollution  measures  ia 
accelerating  ••esearch  in  changed  methods  of  propulsion  which  will 
require  extensive  and  continual  testing  during  the  development  process 

* 

The  automobile  indu8*;y  could  probably  learn  much  from  the 
DOD  about  iperational  testing  and  evaluation.  In  particular,  the  DOD 
appears  to  he  much  more  advanced  in  applying  the  methodologies  of 
operations  rcsearch/systems  analysis  to  problem  solving.  General 
Motors  is  cuirtntly  increasing  its  capabilities  in  this  area. 
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There  are  two  aspects  of  OT&E  in  the  automotive  industry  which 
should  be  more  emphasized  in  the  DOD.  Corporate  headquarters  is 
very  active  in  providing  guidance  to  the  automotive  divisions  and  itself 
performs  relevant  OT&E  on  their  behalf.  Also,  the  automotive 
industry  is  very  active  in  OT&E  designed  to  determine  with  great 
precision  the  characteristics  and  capabilities  of  current  systems. 

The  requirements  for  and  use  of  OT&E  by  the  airlines  parallel 
those  of  the  DOD  much  more  closely  than  do  those  of  the  automotive 
industry.  The  airlines,  similar  to  the  DOD,  have  changing  require¬ 
ments  to  counter  new  threats.  For  the  airlines  the  threat  is  competition, 
which  to  them  is  as  real  a  threat  as  is  the  enemy  to  the  military.  The 
airlines  are  profit  motivated,  and  OT&E  is  primarily  oriented  towards 
increasing  customer  acceptance  and  efficiency  of  operation. 

In  a  very  real  sense,  OT&E  is  continuous  on  all  flights.  Extensive 
data  are  collected  and  maintained  on  all  aspects  of  operations,  partly 
because  of  the  safety  requirements  imposed  by  the  FAA.  These  data 
are  used  by  the  airlines  to  locate  deficiencies,  improve  operating 
and  maintenance  practices,  and  determine  future  requirements  for 
product  improvement. 

In  addition  to  evaluation  of  current  operations,  OT&E  plays  an 
impoitant  role  in  the  procurement  of  new  systems  or  equipment.  In 
general,  there  are  three  separate  methods  of  procurement  in  which 
OT&E  has  a  major  role. 

Firtv,  are  law-risk  "ystems  which  extend  current  capabilities 
but  do  not  challenge  the  state-of-the-art;  these  may  be  larger  or 
smaller  aircraft  of  essentially  the  same  design  as  those  now  in  use. 

Such  a'rcraft  ace  purchased  prior  to  the  first  production.  In  this 
case,  OT&E  is  used  to  confirm  that  the  specifications  and  performance 
guarantees  are  achieved  and  to  determine  optimum  operating  and 
maintenance  practices. 

Second,  are  systems  which  represent  radical  advances  for  which 
there  is  no  reliable  precedent,  such  as  the  Supersonic  Transport,  In 
such  instances,  the  airlines  will  make  no  irrevocable  commitment  for 
purchase  until  prototypes  have  been  produced  and  the  practical  applica¬ 
tion  of  the  concept  has  been  successfully  demonstrated  by  operational 
testing. 
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Finally,  systems  and  equipment  which  have  potential  for  large- 
scale  installation,  such  as  new  navigation  systems  or  baggage  handling 
equipment,  will  be  procured  for  limited  installation  and  test.  This 
pilot  operation  will  provide  information  to  determine  actual  suitability, 
including  customer  acceptance  of  the  system,  and  to  determine  features 
of  the  system  which  require  improvement  prior  to  final  adoption  and 
lar^e-scale  installation. 

Although  there  are  some  similarities  of  the  OT&E  described  to 
that  which  occurs  in  the  DOD,  there  is  much  less  emphasis  in  the 
latter  on  the*  recording  of  data  during  routine  operations.  This  means 
that  OT&E  generally  requires  special  tests.  In  addition,  the  DOD 
feels  compelled  to  take  risks  in  introducing  new  systems  which  airlines 
are  not  permitted  to  take.  In  general,  OT&E  within  the  DOD  has  much 
less  opportunity  to  influence  early  production  systems. 
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GENERAL  DISCUSSION  OF  OT&E  ACTIVITY 
ABOVE  SERVICE  LEVEL 


In  identifying  and  evaluating  the  effectiveness  of  OT&E  at  OSD 
and  JCS  levels,  an  attempt  was  made  to  assess  its  contributions  in 
the  following  aspects: 

1.  Obtaining  information  to  assist  in  determing  require¬ 
ments  for  new  systems  and  materiel. 

2.  Providing  inputs  to  data  bases  needed  in  planning  future 
forces  and  making  choices  among  alternative  future  systems. 

3.  Developing  tactics  and  techniques  for  employing 
systems  already  in  the  inventory. 

4.  Evaluating  the  operational  suitability  of  new  systems, 
as  the  culmination  of  the  RDT&E  process. 

There  are  several  staff  organizations  above  Service  level  that 
are  involved  in  one  or  more  of  these  aspects  of  OT&E.  However,  their 
effort  is  fragmented,  and  none  of  them  has  the  responsibility  of  managing 
or  monitoring  OT&E  as  a  whole.  There  is  no  single  focal  point  for 
information  on  policies,  procedures,  organizations,  and  facilities  for 
•conducting  and  reporting  on  OT&E  within  the  DOD. 

One  reason  why  OT&E  activities  in  OSD  and  JCS  are  so  fragmented 
is  that  there  is  no  consensus  that  a  focal  point  at  that  level  is  required. 
Particularly  in  JCS  there  is  considerable  belief  that  such  matters  are 
properly  left  to  the  individual  Services,  and  that  it  is  possible  and 
desirable  to  rely  on  the  Services  cooperating  and  freely  exchanging 
information  obtained  from  Service  OT&E.  Although  there  is  much  such 
cooperation  and  exchange,  it  would  be  naive  in  the  extreme  to  rely  on 
it  to  provide  OT&E  data  for  high  level  decisions  which  involve  Service 
roles  and  missions.  , 

There  is  no  effective  higher-than-Servicc  level  organization  to 
take  a  broader  point  of  view  towards  OT&E  than  is  possible  at  Service 
level  and  assume  the  responsibility  for  doing  or  having  done  OTfcE 
needed  to  assist  with  important  decisions  which  may  conflict  with  the 
institutional  interests  of  one  or  ,-nur  •  of  the  Services. 
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For  convenience  OT&E  at  OSD  and  JCS  level  can  be  divided'into 
two  categories:  strategic  missile  systems  and  non-nuclear  systems. 

Generally,  OSD  and  JCS  level  supervision  of  OT&E  of  strategic 
missile  systems  is  judged  to  be  adequate.  The  Deputy  Director 
(Strategic  and  Space  Systems),  ODDR&E,  exerts  considerable  influence 
on  OT&E  activities  and  facilities  for  evaluating  overall  strategic  missile 
performance.  In  addition,  this  is  an  area  where  the  Weapons  Systems 
Evaluation  Group  has  made,  and  is  continuing  to  make,  relevant  and 
useful  studies. 

In  marked  contrast,  supervision  of  OT&E  of  non-nuclear  systems 
by  higher-than-Service  agencies  is  manifestly  fragmented,  incomplete 
and  largely  ineffective. 

The  Assistant  Secretary  (Systems  Analysis)  has  been  seriously 
handicapped  by  lack  of  OT&E  data  on  non-nuclear  systems.  In  particular, 
OASD(SA)  has  not  been  able  to  obtain  data  which  reliably  indicate  the 
capabilities  and  limitations  of  such  systems  now  in  the  hands  of  opera¬ 
tional  forces.  Without  such  base-line  data,  it  has  been  extremely 
difficult  to  evaluate  the  worth  of  proposed  follow-on  systems.  OASD(SA) 
would  be  a  major  consumer  of  valid  OT&E  data  if  such  data  were 
available. 

The  Assistant  Director  (OT&E),  ODDR&E,  appears  to  have  the 
principal  responsibility  for  OSD's  interests  in  and  supervision  of 
OT&E  throughout  the  DOD.  This  directorate  has  been  generally 
ineffective  for  the  four  years  it  has  existed.  This  is  partly  attributable 
to  woefully  inadequate  manning.  The  most  important  reason  for  its 
lack  of  influence,  however,  is  its  location.  An  OT&E  organization 
should  not  be  subordinate  to  the  developer.  Even  within  the  developing 
organization  (ODDR&E),  the  Assistant  Director  (OT&E)  is  too 
removed  from  the  decision-making  level.  There  is  very  little 
relationship  between  Assistant  Director  (OT&E)  and  Service  OT&E 
agencies.  Often  the  latter  were  not  even  aware  that  such  an  organiza¬ 
tion  existed  within  the  OSD. 

Two  other  OSD  offices  monitor  and,  in  some  respects,  supervise 
aspects  of  non-nuclear  OT&E,  but  their  functions  duplicate  or  overlap 
those  assigned  to  Assistant  Director  (OT&E).  These  offices  are  the 
Assistant  Director  (Chemical  Technology),  ODDR&E,  who  provides  OSD 


attention  to  joint  OT&E  of  chemical  and  biological  weapons,  and  the 
Defense  Communications  Planning  Group,  which  directs  both  the 
testing  and  operational  employment  of  remote  sensors  and  anti¬ 
infiltration  munitions. 

It  was  disturbing  to  discover  that  the  JCS  has  no  focal  point 
for  OT&E  and  conducts  no  continuing  evaluation  of  what  OT&E  is 
required  to  support  joint  planning.  This  is  a  shortcoming  that  needs 
to  be  remedied.  It  is  evident  that  the  JCS  deals  with  OT&E  on  an  ad 
hoc  basis  as  a  matter  of  policy,  and  the  history  of  such  JCS  initiated 
ad  hoc  efforts  indicates  that  they  have  often  been  very  unproductive. 

In  many  instances,  force  structure  and  war  plans  are  based  on 
estimates  of  capability  and  readiness  that  have  never  been  operationally 
validated.  DDR &E  has  stressed,  without  much  success,  the  need  for 
the  Military  Departments  and  the  Joint  Staff  to  identify  areas  where 
quantification  through  operational  test:ng  would  be  particularly  fruitful. 
DDR&E  has  actually  solicited  proposals  for  such  tests,  but  there  has 
been  no  significant  response.  * 

The  Weapons  Systems  Evaluation  Group  currently  directs  no 
operational  testing  activities  and  has  no  responsibility  in  this  area. 
WSEG  undertakes  operational  evaluations  and  analysis  in  response  to 
directives  fr  jm  both  ODDR&E  and  the  JCS,  but  it  has  no  continuing 
responsibility  in  this  respect.  WSEG  almost  certainly  has  a  greater 
capability  for  OT&E  than  is  being  exploited,  and  this  capability  could 
be  increased  if  the  decision  were  made  tc  do  so.  Assigning  WSEG 
more  OT&E  tasks  should  receive  seriou,  consideration. 


*  DDR&E  Memorandum  for  Secretaries  of  Military  Departments 
and  Chairman,  JCS,  Subject:  Opera!. onal  Testing  to  Evaluate 
Capability  and  Readiness,  dated  13  January  1967  (Attached  as 
Inclosure  5). 
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SPECIFIC  HIGHER -THAN-SER /ICE  OT&E  ACTIVITIES 


This  is  a  description  of  OT&E  in  the  OSD  and  the  JCS,  accompanied 
ly  identification  of  some  problem  areas. 


Office  of  the  Secretary  of  Defense 


Assistant  Director  (OT&E),  Office  of  the  Director,  Defense  Research 
and  Engineering.  The  Assistant  Director  (OT&E)  is  the  only  OSD 
office  specifically  charged  with  OT&E  functions.  This  office  was 
established  in  March  1966,  with  functions  and  responsibilities  pertain¬ 
ing  to  the  testing  and  evaluation  of  weapon  systems  or  materiels  in  . 
service  use  or  in  the  operational  phases  of  testing  and  evaluation. 

It  was  and  is  oriented  chiefly  toward  prpgrams  in  the  area  of  tactical 
warfare. 


The  establishment  of  this  office  reflected  the  feeling  of  then 
Deputy  Secretary  Cyrus  Vance  and  the  DDR&E,  Dr.  Foster,  that 
more  attention  was  needed  at  the  OSD  level  to  matters  of  testing 
and  evaluation  as  they  related  to  the  DOD  research  and  development 
effort.  There  was  the  expressed  intent  to  regularize  the  monitorship 


of  the  interaction  between  R&D  and  any  major  weapon  system  throughout 
it,  life,  hhAJ 


W  DDR&E  Office  Memorandum  No.  4-66,  Subject:  Organizational 
Changes  and  Appointment  of  Personnel  in  ODDR&E  (Attached  is 
Inclosure  I). 

2 /  Secretary  of  Defense  Memorandum  for  Service  Secretaries  (and 
others),  Subject:  Creation  of  Office  for  Operational  Test  and 
Evaluation,  dated  7  July  1966  (Attached  as  Inclosure  2). 

3/  DDR&E  Memorandum,  Subject:  Office  for  Test  and  Evaluation, 
ODDR&E,  dated  July  20,  1966  (Attached  as  Inclosure  3), 

4 /  DDR&E  Memorandum,  Subject:  Functions  of  the  Office  for  Opera¬ 
tional  Test  and  Evaluation,  ODDR&E,  dated  September  21 ,  1966 
(Attache'1  »s  Inclosure  4). 
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Unfortunately,  Jie  successive  functional  statements  of  the 
Assistant  Director  (OT&E)  and  other  OSD  actions  reflect  a  narrowing 
of  responsibilities  from  those  apparently  originally  intended. 

Current  functions  are  as  follows:* 

Applies  OSD  level  attention  to  matters  of  test  and  evaluation 
as  they  relate  to  the  management  of  our  research  and  develop¬ 
ment  effort. 

Ensures  that  facts  exposed  by  operational  test  and  evaluation  in 
the  Services  and  performance  data  derived  from  military  employ¬ 
ment  are  broadly  disseminated  and  exploited  to  speed  research 
and  development,  to  improve  defense  materiel  and  weapons 
systems  capabilities,  and  to  assist  related  Defense  planning. 

Oversees  for  adequacy  and  timeliness  the  operational  test  and 
evaluation  efforts  of  the  Services;  observes  military  exercises, 
maneuvers  and  combat  evaluations  organized  to  test  materiel  in 
the  operational  environment;  and  investigates  reported  deficiencies 
in  operational  weapons  systems,  assisting  in  remedial  actions. 

Performs  ODDR&E  staff  review  and  en  ires  appropriate  action 
on  program  elements  in  the  RDT&E  Pr  ,ram  and  Budget  which 
apply  to  Operational  Test  and  Evaluation 

From  the  beginning  the  Assistant  Direc  or  (OT&E)  functions  have 
applied  primarily  to  no.n-nuclear  OT&E,  and  .  ow  they  also  exclude  the 
joint  chemical/biological  operational  test  arc;  .  The  current  functions 
also  no  longer  include  another  important  area  acting  as  focal  point 
within  OSD  for  information  as  to  policies.,  procedures,  organizations 
and  facilities  for  operational  tests  and  evaluations  of  materiel  and 
systems  in  the  DOD. 

A  further  restriction  on  the  scope  of  effort  has  been  the  focusing  on 
weapon  systems  or  materiel  in  service  use  or  operational  phases  of 
testing  and  evaluation  as  contrasted  to  technical  or  contractor  teste 
occurring  during  development. 


#ODDK&£  Office  Order  No.  22  (Rev.  1),  Subject:  Functional  Statements, 
dated  4  September  1968,  p.  8. 
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The  latter  prlicy  was  informally  modified  in  June  1969  to 
deemphasize  earlier  involvement  with  post-production,  user,  opera¬ 
tional  performance  evaluations  and  to  extend  staff  interest  back  into 
developmental  testing.  However,  there  has  not  yet  been  a  change  in 
formal  functional  statements,  nor  has  there  been  action  by  DDR&E  on 
a  number  of  memos  from  the  Assistant  Director  fOT &£)  pertaining  to 
the  subject. 

The  Assistant  Director  (OT&E)  is  manned  by  five  officers  and 
two  clerical  personnel.  These  officers  do  not  appear  to  have  been 
selected  for  exceptional  experience  and  ability  in  operational  test 
and  evaluation.  Even  if  they  had  such  qualifications,  it  is  difficult 
to  see  how  such  a  small  office  could  be  effective  in  supervising  the 
broad  OT&E  area  on  behalf  of  the  OSD.  Further,  their  placement  in 
DDR&E  subordinates  them  to  the  developer  and  in  effect,  has  the 
developer  evaluating  his  own  product. 

None  of  the  many  Serv  ce  personnel  interviewed  by  the  OT&E 
Tas.k  Group  indicated  receiving  any  assistance  or  gu'dance  from  the 
Office  of  the  Assistant  Director  (OT&E). 

Assistant  Director  (Chemical  Technology)  of  QDDK&E*.  On  21  February 
1963,  the  Assistant  Director  (Chemical  Technology)  assumed  cognizance 
over  the  Deseret  Test  Center  Joint  Chemical/ Biological  Operational  Test 
and  Evaluation  activities  from  the  Assistant  Director  (OT&E).  This 
change  occur  red  when  fhe  Army  Dugway  Proving  Ground  was  merged 
with  the  Deseret  Test  Center.  The  portion  of  the  current  Deseret  Test 
Center  program  pertaining  to  joint  offensive  weapon  OT&E  faces  an 
uncertain  future  in  view  of  the  Presidential  moratorium  in  this  area. 

Deputy  Director  (Strategic  and  Space  Systems).  Test  and  evaluation  of 
nuclear  weapon  systems,  less  live  warhead  testing  in  the  atmosphere, 
is  unde/  the  cognizance  of  the  Deputy  Director  (Strategic  and  Space 
System?)  *  .\i  his  Assistant  Directors  (Strategic  Weapons)  and  (Defensive' 
Systems). 

The  OT&E  of  strategic  missile  systems  appears  to  present  a 
special  situation  which  should  be  treated  separately  and  differently 
from  other  OT&E. 

*ODDE&E  Office  Order  No.  22  (Rev.  1),  Subject:  Functional  Statements, 
dated  4  September  1968.  p.  1  1.  (Extract  attached  as  Inclosure  6). 
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Because  of  the  extreme  cost  and  complexity  of  strategic  missile 
systems  and  of  the  instrumentation  required  to  determine  performance, 
a  separate  independent  OT&E  capability  would  be  too  c  tly  in  both 
dollars  and  trained  personnel. 

The  Assistant  Secretary  (Systems  Analysis).  The  Assistant  Secretary 
(Systems  Analysis)  prepares  Draft  Presidential  Memoranda  on  the 
General  Purpose,  Strategic,  and  other  military  forces;  prepares  force 
structure  comparative  analyses,  including  cost  estimates  of  alternative 
defense  programs;  and  reviews  Service  budget  requests  for  new  weapon 
systems  for  the  Secretary  of  Defense.  The  Assistant  Secretary  (Systems 
Analysis)  has  been  hampered  in  its  analyses  by  the  lack  of  base-line 
data  available  on  performance  of.  current  systems  and  subsystems. 

Joint  Chiefs  of  Staff 

There  is  no  Joint  Staff  agency  specifically  charged  with  OT&E 
responsibility,  and  it  was  the  belief  of  reveral  Joint  Staff  general 
officers  interviewed  that  there  should  not  be  such  a  function  formally 
established  within  the  Organization  of  the  Joint  Chief3  of  Staff. 

The  JC'J  have  from  time  to  time  reacted  to  matters  involving 
CT&E  deficiencies  by  either  charging  the  Weapons  Systems  Evaluation 
Group  with  creating  an  ad  hoc  committee  to  examine  the  problem  or 
perform  a  test  and  evaluation,  or  by  recommending  creation  of  special 
Joint  Task  Forces.  Examples  of  WSEG  activities  are  the  WEXVAL 
exercises  of  the  late  1950's,  a  study  of  OT&E  of  strategic  ballistic 
missiles,  actual  operational  testing  of  the  M-16  rifle  in  Panama,  ana 
an  on-going  study  of  OT&E  of  air-to-air  missiles.  Examples  of  joint 
task  forces  are  Joint  Task  Force  Two  and  Joint  Task  Force  Eight. 


Generally  these  ad  hoc  committees  and  task  forces  were  reactive, 
were  organized  long  after  a  major  problem  was  recognized,  and  took 
considerable  time  to  get  started.  It  has  been  observed  chat  joint  task 
forces  tend  to  disappear  under  later  budget  .and  manpower  pressures, 
sometimes  without  major  accomplishments. 

Joint  Task  Force  Two,  established  1964  to  examine  low  level 
penetration  problems,  was  disestablished  m  1963  with  "ery  little  ' 
show  for  approximately  $80  million  dollars  expended. 
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The  Deseret  Test  Center,  established  to  conduct  joint  chemical 
biological  operational  testing,  has  teen  another  costly  venture  and 
nc'w  faces  an  uncertain  future* 

Joint  Task  Force  Eight  was  hurriedly  created  in  1961  to  enable 
resumption  of  atmospheric  nuclear  testing*  It  will  be  disestablished 
in  July  1970, 

Weapons  Systems  Evaluation  Group 

(WSEG  is  actually  assigned  to  and  funded  hv  ODDRkE:  however, 

it  receives  about  75%  of  its  tasks  from  the  jCSj 

History.  In  December  1943,  Secretary  of  Defense  Forrestal  directed 
establishment  of  the  Weapons  Systems  Evaluation  Group,  reporting 
administratively  to  the  Research  and  Development  Board  and  providing 
guidance  to  both  the  RD3  and  to  JCS. 

WSEG  was  to  apply  techniques  of  "Operations  Research"  to  the 
evaluation  of  the  relative  military  worth  of  different  weapon  systems. 
The  term  "Operations  Research"  as  used  here  is  the  same  as  opera¬ 
tions  analysis  -  a  form  of  evaluation  that  deals  with  weapon  systems, 
tactical  doctrine,  methods  of  warfare  and  the  like. 

It  had  been  understood  when  WSEG  was  formed  that  it  would  be 
placed  directly  under  the  JCS  at  the  end  of  one  year.  However,  this 
transfer  was  opposed  later  because  of  a  feeling  that  it  must  be  free 
and  independent  to  express  its  opinions  without  fear  or  favor  and  able 
to  undertake  studies  it  deemed  important. 

From  1954  until  1962,  the  WSEG  mission  was  as  follows; 

1,  To  provide  the  Department  of  Dsfense  with  comprehensive, 
objective,  and  Independent  analyses  and  evaluations  under  projected 
conditions  of  war,  which  will  include  but  will  not  necessarily  be  con¬ 
fined  to: 


a,  Present  and  future  weaoon  systems. 

b.  The  influence  of  present  and  future  weapon  systems  upon 
strategy,  organization,  and  tactics. 
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c.  The  comparative  effectiveness  and  costs  of  weapon 

systems. 

2.  To  make  available  to  the  Department  of  Defense  timely  advice 
and  assistance  to  aid  decisions  in  the  allocation  of  resources  for  develop¬ 
ment  of  the  most  effective  combination  of  weapon  systems. 

WSEG  also  had  the  responsibility  for  undertaking  such 
studies  as  the  Group  itself  might  decide  to  initiate  on  the  grounds 
of  reference  to  current  and  projected  work  of  the  Group. 

The  Institute  for  Defense  Analyses  (IDA)  was  created  in  1956 
to  provide  direct  contract  support  to  WSEG. 

Current  Status.  The  Weapons  Systems  Evaluation  Group  is  a  Department 
of  Defense  organization  under  the  administrative  direction  of  the  Director- 
of  Defense  Research  and  Engineering  within  the  Office  of  the  Secretary  of 
Defense  (OSD). 

Since  1962,  WSEG  has  been  charged  with  conducting  operational 
analyses  and  evaluations  for  the  Joint  Chiefs  of  Staff  (JCS)  and  the 
Director  of  Defense  Research  and  Engineering  (DDR&E),  and  other 
elements  of  the  Office  of  the  Secretary  of  Defense  as  authorised  by 
the  Secretary  of  Defense;  and  with  participation  in  and  supervision 
of  such  WSEG  study  contracts  with  civilian  or  other  government  agencies 
as  may  be  required  in  discharge  of  its  mission. 

WSEG  is  a  composite  organization  directed  by  a  Lieutenant  General 
or  Vice  Admiral  and  staffed  by  about  50  senior  officers  drawn  from  the 
several  Services.  Civilian  analysts  for  the  projects  come  from  the  con¬ 
tractor  (principally  IDA)  with  wiiorr  WSEG  contracts  for  a  given  project. 

WSEG  has  recently  conducted  one  Operational  Test  and  Evaluation  -- 
that  of  the  M-16  rifle  in  Panama  in  1967/1968.  For  the  most  part,  its 
efforts  have  consisted  of  operational  analyses  and  operational  evaluations 
using  as  inputs  data  obtained  from  the  Services  and  their  test  agencies. 
WSEG  has  not  participated  in  the  Service  operational  tests  of  new  weapon 
systems  at  the  stage  where  production  might  be  affected  by  the  findings. 
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WSEG  evaluations  today  are  reactive  and  generally  take  several 
months  to  organiae  after  a  major  problem  prompts  the  JCS  or  an  OSD 
office  to  request  a  study.  The  present  WSEG  mission  appears  on  piper 
to  be  considerably  reduced  in  scope  and  initiate  from  what  it  was  in  the 
195C's. 
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OFFICE  OF  DIRECTOR  OF  DEFENSE  RESEARCH  l  ENGINEERING 
WASHINGTON,  D.  C. 


si  m 

s:  s 


March  7,  1966 


OFFICE  MEMORANDUM  NO.  4-66 

SUBJECT.  Organizational  Changes  and  Appointment  of  Personnel 
m ODDR&E 


Effective  21  February  1966,  a  new  Deputy  Directorate  for  Chemistry 
and  Materials  was  established  in  ODDP. i-E.  The  Offices  of  the 
Assistant  Director  for  Materials  and  for  Chemical  Technology  were 
reassigned  from  the  Deputy  Director  (Research  and  Technology) 
to  the  new  Deputv  Directorate.  Dr.  Donald  M.  MacArthut  w«® 
assigned  as  Deputy  Director  (Chemistry  and  Materials)  on 
21  February  1966. 

inactive  7  March  1966,  the  position  of  Assistant  Director  (Ad¬ 
ministration  and  Management)  is  abolished  and  the  Ofnce  of  Assistant 
Director  (Operational  Test  and  Evaluation)  is  established,  with 
Rear  Admiral  Vincent  P.  de  Poix,  USN,  assigned  as  the  Assistant 
Director.  The  new  office  will  function  under  the  Deputy  Director 
(Administration  and  Management).  Functions  and  .responsibilities 
of  the  new  office  will  pertain  in  test  and  evaluation  of  weapons 
systems  or  materials  in  service  use  or  in  the  operational  phases 
of  teat  and  evaluation  as  contrasted  to  technical  or  contractor  te6t, 
and  will  be  oriented  chiefly  toward  programs  in  the  area  of  tactical 
warfare. 

Effective  7  March  1966,  Mr.  Edgar  G.  Shelor,  Jr. ,  is  designated 
Assistant  Director  (Communications  and  Electronics), 

Effective  21  March  1966,  the  office  of  Assistant  Di  -error  (Plans 
and  Policy)  wilt  be  terminated.  *  unctions  of  that  office  pertaining 
to  planning,  program  objectives,  program  guidance  and  program 
analysis  will  be  transferred  to  the  Office  of  Program  Review  and 
Coordination,  Functions  pertaining  to  communications  with  industry 
and  industry  groups  (guidelines,  brieftrgs,  etc.)  will  be  transferred 
to  the  office  of  the  Assistant  Director  (Engineering  Management). 

Effective  21  March  1966,  Mr.  Paul  Sturm,  A- « i-itarit  Director  (Elan, 
and  Policy)  is  redesignated  as  Special  Assistant  (Plans  and  Policy) 
reporting  to  the  Deputy  Director  (Administration  and  Management). 


^ep;'°SabV 
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V/m,  J.  Ely 
Lt.  General,  USA 
Deputy  Director 
(Administration  &  Management) 
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THE  SECRETARY  OF  DEFENSE 
WASHINGTON 


7  JUL  1966 


MEMORANDUM  FOR  Secretaries  of  the  Military  Departments 
Chairman  of  the  Joint  Chiefs  of  Staff 
Director  .-,f  Defense  Research  and  Engineering 
Assistas..  Secretaries  of  Defense 
Assistants  to  the  Secretary  of  Defense 
Directors  of  the  Defense  Agencies 

SUBJECT:  Creation  of  Office  for  Operational  Test  and  Evaluation 


A  new  office  for  Operational  Test  and  Evaluation  has  recently  been 
formed  within  the  Office  of  the  Director  of  Defense  Research  and 
Engineering.  The  establishment  of  this  office,  under  an  Assistant 
Director,  reflects  my  f=eiiag  and  that  of  the  Director  of  Defense 
Research  and  Engineering  that  more  attention  is  needed  at  the  OSD 
level  to  matters  of  test  and  evaluation  as  they  relate  to  cur  research 
and  development  effort. 

While  it  is  well  recognized  that  there  is  almost  invariably  an  involve  - 
ujent  between  research  and  development  and  any  major  weapon  system 
throughout  its  life,  the  monitorship  of  this  interaction  has  not  before 
been  regularized  within  OSD.  As  implied  in  the  name,  it  is  intended 
that  the  activities  of  the  new  office  will  be  devoted  primarily  to  matters 
of  test  and  evaluation  as  they  relate  to  the  user  rather  than  the  developer: 
that  is,  with  phases  of  the  life  of  systems  or  equipment  subsequent  to 
contractor  or  technical  test  and  evaluation. 
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INCLOSURE  2 


OFFICE  OF  THE  DIRECTOR  OF  DEFENSE  RESEARCH  AND  ENGINEERING 

Washington,  D.  C. 

July  20,  1966 


MEMORANDUM  FOR  DEPUTY  DIRECTORS 

ASSISTANT  DIRECTORS 
OFFICE  DIRECTORS 

SUBJECT:  Oftice  for  Test  and  Evaluation,  ODDRE 

REFERENCES:  a)  ODDRE  Office  Memo  No.  4-66,  March  7,  1966 

b)  Memo  from  Deputy  SECDEF,  Mulli -addressees, 
subject:  Creation  of  Office  for  Operational  Test 
and  Evaluation,  July  7,  1966 


As  stated  in  the  above  two  references,  the  office  for  Operational  Test 
and  Evaluation  in  ODDRE  was  formed  to  give  more  attention  at  OSD  level 
to  matters  of  test  and  evaluation  as  they  relate  to  our  research  and 
development  effort.  More  specifically,  it  is  intended  that  the  activities 
of  thi3  office  be  devoted  primarily  to  test  and  evaluation  matters  relating 
directly  to  the  user  rather  than  •.he  developer.  The  functions  and 
responsibilities  of  this  office  then,  will  pertain  to  weapon  systems  or- 
materials  in  service  use  or  operational  phases  of  test,  and  evaluation  as 
contrasted  to  technical  or  contractor  tests  occurring  during  development. 
Thus,  other  ODDRE  offices  will  continue  to  exercise  ineir  responsibilities 
for  developmental  tests. 

Experience  has  shown  that  in  test  and  evaluation  frequently  there  is  no 
clear-cut  line  separating  the  area  of  concern  of  the  user  from  that  of 
the  developer.  This  absence  of  definition  makes  it  essential  that  the 
ODDRE  offices  concerned  maintain  the  closest  coordination  with  the 
Office  of  Operational  Test  and  Evaluation  to  insure  the  integrity  of  the 
test  and  evaluation  effort. 

An  understanding  of  the  functions  of  the  new  office  for  Operational  Test 
and  Evaluation  will  assist,  among  other  things,  such  coordination. 

For  this  reason  these  functions  as  presently  visualized  are  listed  below: 

I,  Investigates  reported  deficiencies  in  weapons  systems  which 
are  deployed  or  have  been  accepted  for  service  use,  and  insures  that 
appropriate  action  is  taken  to  diagnose  problems  and  identify  RfcD  action 
needed. 
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2.  Monitors  certain  major  field  or  fleet  exercises  or  maneuvers 
of  the  Services  for  the  appearance  of  operational  or  technical  problems 
which  require  some  RDT&E  action  for  correction. 

3.  Collaborates  with  the  Services  as  needed  in  devising  tests  to 
determine  actual  combat  readiness  of  weapons  systems  which  may  come 
under  question. 

4.  Reviews  plana  for  and  results  of  the  operational  tests  and 
evaluations  conducted  by  the  Services  on  major  new’  weapons  systems  to 
determine  the  adequacy  of  these  systems  for  operational  employment. 

5.  Reviews  the  programs  of  the  Services  for  research  and 
development  on  major  items  of  training  equipment  auu  devices  Intended 
to  insure  combat  readiness  of  our  operational  forces. 

G.  Collaborates  with  the  Services  to  insure  the  “’’liability  of 
material  needed  to  support  operational  training  in  major  weapons  systems. 

7.  Functions  as  ODDRE  action  office  in  connection  with  directing 
the  plans  and  activities  of  Joint  Task  Force  Two  and,  as  assigned,  any 
other  joint  task  force  or  project  which  has  a  mission  in  operational  test 
and  evaluation. 

9.  Acts  as  the  focal  point  within  OSD  of  information  pertaining  to 
policies,  procedures,  organizations  and  facilities  for  operatioi«al  test 
and  evaluation  of  material  and  systems  in  the  Department  of  Defense, 
primarily. those  concerned  with  tactical  warfare.  Initiates  such  policy 
or  guidance  as  may  be  required  i-i  connection  with  the  above. 
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WM,  J.  ELY  ’  1  , 

Lt.  General,  U.  S.  Army 
Deputy  Director  , 
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OFFICE  C?  TIE  DIRECTOR  OF  DEFENCE  RESEARCH  AKU  EKC-IJIEERINO 
WASH! NOTCH,  D.  C.  20301 

September  21,  1^36 


KSiSSAKDWJ  FOR  DEruTT  DIRECTORS 

ASSISTANT  DIRECTORS 
OFFICE  DIRECTORS 

SUBJECT:  Functions  of  the  Of  floe  for  0-  era!  ViihI  Test  end  Evelu&tion, 
0DDRA5 

REFERENCES:  a)  CDDR&K  Office  Wprnn  Ho,  “-oo,  dated 

natch  7,  i960,  subj;  Organizational  Changes 

mTiu  AppClUwMuiw  ox  £  tcx  ouiUiex  1U  C)UUZ\olE* 

b)  Memo  from  Deputy  SECDEF  to  multiple 
addressees,  dated  July  7.  1966,  suhj:  Creation 

c)  Memo  from  DD(ASM),  ODDR&E,  to  above 
addressees,  dated  July  20,  1966,  subj:  Office 

.  •  •  for  Test  and  Evaluation,  IDDR&E 


The  Office  of  Operational  Test  and  Evaluation  has  been  established  to  assist 
the  Director  of  Defense  Research  and  Engineering  to  apply  the  results  of 
operational  test  and  evaluation  to  the  RED  process.  Immediate  objectives  of 
this  application  are  to  orient  RAD  toward  equipment. deficiencies  or  new 
requirements  found  by  test  and  evaluation  or  exposed  by  service  experience; 
to  add  speed  and  precision  to  the  K&D  process  vnere  the  results  of  oper¬ 
ational  test  and  evaluation  may  be  exploited;  and  to  identify  for  action 
those  fixes  and  improvements  that  significantly  enhance  capability  or  extend 
service  life  of  weapons,  weapons  systems  end  Defer.™  materiel.  A  longer 
range  objective  is  the  study  of  test  and  evaluation  data  for  the  purpose  cf 
identifying  technical  criteria  that  should  be  applied  in  force  structure 
planning.  Related  also  is  the  function  ci  coordinating  the  development  ar.d 
use  of  Service  test  and  evaluation  facilities,  primarily  those  concerned 
with  tactical  warfare,  to  enhance  efficiency  in  the  use  of  resources  and 
time. 

It  is  intended  that  the  activities  of  this  office  be  devoted  primarily  to 
test  and  evaluation  matters  relating  directly  to  the  user  rather  than  the 
developer.  The  functions  and  responsibilities  of  this  office  will  therefore 
pertain  to  weapon  systems  or  materiel  in  service  use  or  operational  phases 
of  test  and  evaluation  as  contrasted  to  Cevimica.1  or  contractor  tests 
occurring  during  development.  Other  ODDR&E  offices  will  continue  to 
exercise  their  responsibilities  for  developmental  tests. 

Experience  has  shown  that  in  test  and  evaluation  frequently  there  is  no  clear- 
cut  line  separating  the  area  of  concern  of  the  user  from  hat  of  the  developer. 
This  absence  of  definition  makes  it  essential  that  the  ODDIi&E  offices  concerned 
maintain  the  closest  coordination  with  the  Office  of  Operational  Test  and 
Evaluation  to  insure  tne  integrity  of  the  test  and  evaluation  effort. 
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Reference  c) ,  now  superseded  by  this  memcrandu,  advised  addressees  of  functions 
as  then  visualized  for  the  office  for  Operational  Test  and  Evaluation,  CDDR&3. 
Although  final  details  will  continue  to  be  developed,  the  functions  currently 
assigned  are  as  follows; 

1,  Investigates  report«i  i<*f iciencies  in  weapons  systems  already  deployed 
or  accepted  for  service  use,  and  insures  appropriate  action  is  tales  to  diagnose 
and  apply  remedial  research  and  development. 

2.  Mcnitors  selected  major  joint  or  Service  field  or  fleet  exercises  or 
maneuvers  to  identify  operational  Or  technical  problems  requiring  IfflT&S 
corr*<**<v*  action* 

3*  Collaborates  vith  the  Services  ;*.£  weeded  in  devising  to  assure 

eonibat  readiness  of  or  to  identify  deficiencies  in  weapons  systems. 

4.  Reviews  plans  for  and  results  of  operational  tests  and  equipment  where 
such  tests  and  evaluations  are  designed  to  determine  the  adequacy  of  the  tested 
items  for  .vnerationel  use. 

5.  In  coordination  with  other  ODDR&E  offices,  reviews  for  completeness, 
adequacy  and  timeliness  the  research  and  development  programs  of  the  Services 
on  major  items  of  equipment  and  training  devices  intended  tc  insure  combat 
readiness  of  operational  forces. 

6.  Collaborates  with  the  Services  to  insure  the  suitability  of  material 
needed  to  support  operational  training  in  major  weapons  systems. 

7.  Serves  as  CDDR&E  actiou  office  on  matters  pertaining  to  Joint  Task 
Force  T«o  ar.d,  as  assigned,  other  joint  task  forces  or  projects  with  a  mission 
of  operational  test  and  evaluation. 

8.  Performs  ODDR&E  staff  review  and  takes  appropriate  action  on  progress 
elements  in  the  RDT&E  Program  and  Budget  which  apply  to  operational  test  and 
evaluation, 

9.  Acts  as  the  focal  point  within  OSD  for  information  as  to  policies, 
procedures,  organizations  and  facilities  for  operational  test  and  evaluations 
of  materiel  and  systems  in  the  DOD. 

10.  Initiates  policy  or  guidance  as  required  to  promote  coordination  among 
the  Services  in  use  and  development  of  instrumentation,  equipment,  facilities 
and  methodologies  for  operational  test  and  evaluation. 
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KARVIi;  L.  McKICKLE 
Lt.  General,  U.  S.  Air  Force 
Deputy  Director  (Admini Deration, 
Evaluation  and  Management) 
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OFFICE  OF  THE  DIRECTOR  OF  DEFENSE  RESEARCH  AND  ENGINEERING 

WASHINGTON,  D.  C.  20301  _ 


13  JAN  1967 


MEMORANDUM  FOR:  Secretaries  of  the  Military  Departments 

Chairman  of  the  Joint  Chiefs  of  Staff 


SUBJECT:  Operational  Testing  to  evaluate  capability  rr-adiness 


Our  Force  Structure  and  War  Plans  a~e,  in  many  cases,  based  on 
estimates  of  capability  and  readinr  .,  at  have  never  beer.  operationally 
validated.  The  studies  which  gre^  '  uflucncc  our  Force  Level 
decisions  frequently  contain  a  sue  sion  of  performance  estimates, 
the  variation  of  which  could  significantly  change  our  opinions.  I  am 
particularly  concerned  that  the  results  we  obtain  during  acceptance 
testing  at  the  end  of  the  materiel  Development  phase  are  more 
optimistic  than  wo  can  expect  when  the  materiel  is  in  the  hands  of 
normal  Field  Forces. 

*  • 

Modern  instrumentation  and  computer  technology  offer  tools  of 
prime  utility  for  measuring  our  real  operational  capability.  They 
could  per  for  nr,  invaluable  service  ir.  this  role  it  there  existed  a 
systematic  program  of  operational  tests  aimed  at  establishing  the 
performance  factors  of  Our  equipment  in  the  hands  cf  the  actual  user. 
With  these  factors  for  inputs,  planning  then  can  be  based  on  scientific 
measurements  rather  than  estimates, 

I  is  desired  therefore  that  the  Departments  and  the  Joint  Staff  examine 
the  projee'ed  Force  Structure  and  planning  to  identify  areas  where 
quantification  through  operational  testing  would  be  particularly  fruitful 
fruitful.  I  would  welcome  proposals  for  initial  candidate  tests  for 
this  program  in  the  near  future  so  that  we  can  commence  to  eliminate 
uncertainties  at  the  base  of  our  planning. 


John  S.  Foster,  Jr. 
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(Ex*-' act  from  ODDR&E  Office  Order  No.  22  (Revision  1),  Subject:  Functional 
Statements,  dated  4  September  1968,  p.  14) 


ASSISTANT  DIRECTOR  (CHEMICAL  TECHNOLOGY) 

Has  cognizance  of  the  DOD  RDT&E  programs  in  the  following  areas: 


a. 

Chemical  warfare 

b. 

Biological  warfare 

c. 

Materials  technology 

* 

d. 

Life  sciences  (except  Social  and  Behavioral  Sciences) 

*. 

Medical  sciences 

f. 

Bioastronautics 

> 

Reviews  the  Service  submissions  for  these  programs  for  budget  and  i 

apportionment  purposes  to  assure  that  they: 

a.  have  priority  consistent  with  military  needs  and  r 

requirements, 

b.  represent  reasonable  and  realistic  technical 

approaches,  and 

c.  do  not  contain  needless  or  wasteful  duplication  of  ,  < 

effort. 

Monitors  the  Service  programs  during  the  year  of  execution  to  achieve 
rapid  exploitation  of  technological  opportunity,  cancellation  of  efforts  which 
prove  less  desirable  than  originally  thought  and  such  other  managerial  ac-  j 

tions  as  may  be  done  to  maximize  return  on  R&D  investment.  L 


[J 
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HISTORY  OF  ARMY  TESTING 


The  heart  of  the  Army  Operational  Test  and  Evaluation  (CT&E), 
prior  to  quantity  production  of  weapon  systems  has  historically  been 
located  at  the  Six  Service  Test  Boards  which  are  currently  part  of  the 
U.  S.  Army  Test  and  Evaluation  Command  (TECOM),  a  major  subordinate 
command  of  the  U.  S.  Army  Materiel  Command  (AMC),  [See  Inclosure  1). 
Prior  to  the  establishment  of  AMC  in  mid  1962,  the  Technical  Services 
vere  responsible  for  engineering  tests,  which  determined  the  technical 
performance  and  safety  acceptability  under  controlled  test  conditions, 
and  in  many  cases  had  established  separate  facilities  for  this  mission. 

The  Service  Test  Boards  were  assigned  to  the  U.  S.  Army  Continental 
Army  Command  (CONARC)  and  conducted  their  OT&E  in  a  manner  similar 
to  current  testing  under  TECOM  to  determine  the  overall  suitability  of 
the  item  of  equipment  for  operational  use  within  the  Army.  The  estab¬ 
lishment  of  AMC  consolidated  in  TECOM  the  independent  test  facilities 
of  the  Technical  Services  and  the  CONARC  boards.  At  the  time  TECOM 
was  formed  u  was  felt  that  both  the  developer /producer  and  the  user 
had  an  interest  and  a  need  for  the  results  of  engineering  tests  and 
service  tests.  The  user's,  as  compared  to  the  developer/producer 's 
interest  in  service  test  results,  was  considered  to  be  predominant. 
Consolidation  of  ihe  responsibility  for  both  engineering  and  service 
tests  within  TECOM  was  expected  to  result  in  a  more  efficient  and 
responsive  organization  which  could  provide  a  basis  for  recommending 
possible  trade-offs  between  technical  capability  and  operational  require¬ 
ments.  In  1962,  the  U.  S.  Army  Combat  Developments  Command  (CDC) 
was  establish  ;d  which  provided  an  org.  nization  with  overall  responsi¬ 
bility  for  representing  the  user  and  a  means  for  achieving  better  user 
field  teste,  experiments  and  evaluations. 

During  the  period  1965-67,  the  Army  conducted  a  very  extensive 
in-depth  review  of  the  test  and  evaluation  process  as  a  result  of  the 
study  of  Army  Test  and  Evaluation  (SATE)  and  the  DA  Board  of  Inquiry 
on  the  Army  Logistics  System  (Brown  Board).  Both  SATE  and  the 
Brown  Board  made  careful  distinction  between  the  test  and  evaluation 
function,  or  more  specifically  between  materiel  testing  and  operational 
evaluation.  Materiel  testing  was  considered  to  be  a  process  by  which 
data  is  accumulated  to  serve  as  a  basis  for  assessing  the  degree  to 
which  a  materiel  item  or  system  meets  or  fails  to  meet  the  technical 
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or  performance  specifications  toward  which  it  was  designed  and  built. 
Operational  evaluation  was  felt  to  be  a  subjective  determination  of  the 
utility,  i.  e.  ,  the  military  operational  value  to  the  user,  of  the  item  or 
system  when  measured  against,  the  threat  analysis  and  future  require¬ 
ments  of  concept,  doctrine,  environment,  organization,  skills, 
supportability  and  obsolescence.  Evaluation  was  considered  to  be 
broad  in  scope  and  included  testing  as  one  of  its  elements. 

Using  the  above  definitions  of  materiel  testing  and  operational 
evaluation,  the  SATE  recommended  eight  improvement  actions,  of 
which  six  were  approved  by  the  Chief  of  Staff  for  implementation.  The 
recommendation  pertaining  to  where  the  Service  Test  Boards  and 
TECOM  should  be  relocated  organizationally  wilhi'  the  Army  and  the 
one  replacing  ET  &  ST  with  an  integrated  Development  Acceptance  Test 
were  not  approved.  In  1967,  the  Brown  Board  Report  agreed  with 
SATE  that  the  evaluation  function  was  being  slighted,  but  rather  than 
recommending  any  reorganization  of  TECOM,  the  Brown  Board 
recommended  major  changes  to  Army  Regulations  to  emphasize: 

(1)  the  evaluation,  process;  (2/  test  support  responsiveness  of  TECOM 
to  the  user  (CDC),  traine*-  (CCNARC),  logistics  (Logistics  Doctrine, 
Systems  and  Readiness  Agency  -  LDSRA)  as  well  as  the  developer. 

These  recommendations  were  included  in  new  regulations  such  as 
AR  705-5  (Army  R&D),  dated  April  1968,  AR  '’0-10  (Test  and 
Evaluation  during  R&D  of  Materiel),  dated  December  1968,  AR  71-6 
(Type  Classification/Reclassification  of  Army  Marcriel),  dated 
November  1969,  and  DA  Pamphlet  11-25  (Life  Cycle  Management 
Model  for  Army  Systems),  dated  October  1968.  These  new  regulations 
now  require  such  things  as  development  of  a  Coordinated  Test  Program 
(CTP)  as  part  of  the  System  Development  Plav  they  require  user  (CDC), 
trainer  (CONARC)  and  logistician  (LDSRA)  participation  in  development 
or  the  CTP;  they  require  user  (CDC)  approval  ol  the  Service  Test 
Plan;  and  require  user,  trainer,  and  logistician  participation  in  the 
five  In-Process  Reviews  (IPR)  conducted  by  the  project  manager  and 
the  subsequent  System  Status  Evaluations  (SSE)  held  by  CG,  USAMC. 

It  should  be  noted  that  the  new  procedures  described  above  have  only 
been  published  within  the  last  1-2  years  and  the  full  impact  of  many 
of  these  changes  have  not  really  been  felt  on  prior  development 
programs. 

In  response  to  Congressional  critic'sm  of  inadequate  testing  of 
new  weapon  system?  and  guidance  from  OSD  that  "ordinarily,  full 
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production  of  a  system  wiil  be  approved  only  after  operational  tests 
and  field  experimentation  have  demonstrated  acceptable  effectiveness, 
compared  to  existing  systems,  "  the  Chief  of  Staff  directed  a  review 
of  Army  Operational  Test  and  Evaluation  procedures  in  June  1969. 

This  review  was  rather  extensive  and  involved  the  senior  General 
Officers  from  the  DA  Staff,  AMC,  CDC.  CONARC,  and  TEC OM. 

Some  of  the  key  recommendations  were: 

1.  That  the  current  Service  Test  should  be  expanded  to 
include  more  OT&E  to  be  completed  prior  to  the  decision  on 
production,  (new  test  to  be  called  Operational  Service  Test). 

2.  That  additional  test  items  must  be  made  available 

for  the  new  Operational  Service  Test  (OST)  to  allow  for  testing 
a  small  tactical  element,  (e.  g. ,  tank  platoon). 

3.  That  the  military  utility  of  a  new  item  of  equipment 
must  be  assessed  earlier  in  the  development  cycle  using 
prototypes  whenever  possible  (to  be  called  Developmental 
Suitability  Test). 

4.  That  no  changes  be  made  in  the  Army's  testing  organiza¬ 
tion  or  procedures  except  to  raise  the  caliber  of  test  personnel 
at  the  Service  Test  Boards  (i.  e. ,  Capt,  Major,  ETC  rather 
than  Lts),  and  provide  additional  test  personnel  when  required 

to  conduct  OST. 

5.  That  a  more  direct  involvement  of  Center  Commanders 
(Armor  Center,  infantry  Center,  etc. )  should  be  accomplished 
during  OT&E. 

6.  That  OT&E  reports,  with  unfillered  comments  of  user, 
trainer,  logistician,  and  devel':  per,  be  forwarded  directly  to 
DA  staff  after  completion  of  these  tests. 

Work  is  underway  to  revise  AR  70-10  in  accordance  with  the 
recommendations  of  the  OT&E  study  discussed  above.  In  addition, 
more  emphasis  is  being  given  to  such  things  as  ensuring  that  good 
Coordinated  Test  Programs  are  being  prepared,  that  tests  are  wcU 
designed  to  include  operational  aspects,  and  that  results  of  tests  and 
user  comments  arc  availaole  for  review  during  the  decision  process. 
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On  1  October  1969,  the  Army  formally  established  a  new  OT&E 
type  test  organization  at  Fort  Hood,  Texas,  called  the  U.  S.  Army 
Project  Mobile  Army  Sensor  Systems  Test  Evaluation  and  Review 
(MASSTER).  The  mission  of  Project  MASSTER  will  be  to  insure 
development  of  an  optimum  battlefield  intelligence  gathering  system 
as  part  of  the  Army's  integrated  area  control  system  and  provide  for 
ihe  evaluation  of  Army  surveillance,  target  acquisition,  and  night 
observation  matters.  Project  MASSTER  is  unique  in  the  Armv.  It 
wili  run  both  materiel  and  troop  evaluation  u#ing  representative 
local  troop  units  which  will  normally  be  found  in  a  Brigade-size 
organization.' 

With  the  formation  of  CDC  in  1962,  the  Army  strengthened^  its 
capability  to  conduct  good  OT&E  on  selected  items  of  hardware  prior 
to  production  and  on  all  equipment  being  issued  to  the  Army  in  the 
field.  During  the  past  eight  years  there  has  been  increased  emphasis 
on  field  evaluations  and  experimentation  at  the  U.  S.  Army  Combat 
Developments  Command  Experimentation  Command  (CDCEC)  located 
at  Fort  Ord,  California.  CDCEC  has  a  dedicated  Brigade-sized 
Army  unit  which  provides  direct  support  for  all  field  experiments. 
When  additional  test  units  are  needed;  e.  g. ,  a  helicopter  company 
to  run  OT&E,  these  are  moved  to  CDCEC  for  the  duration  of  their 
test. 

ARMY  STAFF  ORGANIZATION  FOR  OT&E 

On  the  Army  staff,  the  overview  of  OT&E  is  provided  during 
the  materiel  development  process  (i.  e.  ,  Operational  Service  Test) 
by  the  Management  and  Evaluation  Division,  Office,  Chief  of  Research 
&  Development  (OCRD).  This  new  Division  was  formed  with  a  Test 
and  Evaluation  Branch  at  about  the  same  time  that  the  Office  of 
Assistant  Director  (OT&E)  was  established  in  ODDR&E.  The  many 
changes  recommended  for  improving  the  weapons  system  acquisition 
process  which  flowed  from  the  SATE  and  Brown  Board  reports  were 
taken  by  this  jw  Division  and  used  to  revise  Army  Regulations  which 
have  been  published  within  the  last  two  years.  The  Assistant  Chief  of 
Staff  for  Force  Development  (ACSFOR)  has  a  counterpart  to  OCRD 
which  is  the  System  Management  Division.  This  Division  monitors 
the  user  field  tests,  experiments  and  evaluations  conducted  by  CDC 
and  is  the  action  office  on  the  DA  staff  for  all  type  classification 
actions.  A  Very  close  relationship  is  maintained  between  these  two 
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Divisions  in  OCRD  and  ACSFOR  which  insures  maximum  visibility 
of  test  results  at  the  time  when  a  decision  has  been  requested  on  type 
classification  of  materiel. 

The  day-to-day  staff  work  required  for  an  individual  major  weapons 
system  i3  carried  out  by  the  DA  Systems  Staff  Officer  (DASSO)  in  ACSFOR 
and  the  Project  Monitor  assigned  to  the  hardware  Directorates  within 
OCRD. 

The  System  Manager  for  Surveillance  Target  Acquisition  and 
Night  Operations  (STANO),  who  is  assigned  to  the  Office  of  the 
Assistant  Vice  Chief  of  Staff  (AVC/S),  supervises  the  OTfcE  which 
will  be  performed  by  Project  MASSTER  at  Fort  Hood,  Texas. 

The  System  Manager  for  the  SAFEGUARD  missile  system  also 
works  directly  for  the  AVC/S  and  has  a  separate  field  command  for 
development  &  testing  of  SAFEGUARD. 

THE  ARMY  TESTING  CYCLE 

The  Army  method  of  conducting  OTScE  has  undergone  extensive 
review  during  the  past  year  in  an  attempt  to  identify  changes  that  might 
be  made  to  our  test  procedures  and  regulations  which  would  enable 
us  to  obtain  better  equipment.  Some  potential  areas  for  improvement 
have  been  identified  and  arc  being  incorporated  in  a  revision  to  AR  70-10 
(T&E  During  R&D  of  Materiel).  Inclosure  2  contains  the  new  OCRD 
Proposed  Weapons  Systems  Validation  Process  which  has  been  drawn 
up  to  provide  increased  emphasis  on  (1)  early  OT&E  thru  the  Develop¬ 
mental  Suitability  Test,  (2)  obtaining  early  information  as  to  any  problem 
areas  which  may  develop  with  the  weapons  system  as  a  result  of 
unusual  stress  found  under  combat  conditions,  and  (3)  ensuring  direct 
involvement  of  the  user,  trainer  and  logistician  throughout  the  testing 
cycle. 

Before  discussing  the  tests  conducted  on  Army  materiel,  a 
discussion  of  the  procedures  for  type  classifying  equipment  would  be 
appropriate.  Life  Cycle  Phases  and  related  type  classification 
designations  are  discussed  below  (See  Inclosures  3  and  4). 


1.  Development  category.  This  category  contains  items 
that  have  not  yet  qualified  for  adoption.  The  three  designations  within 
the  development  category  are: 

a.  Development  type  (DT).  An  item  being  developed 
or  tested  to  meet  an  approved  qualitative  materiel  requirement  (QMR), 
small  development  requirement  (SDR),  other  DA-approved  requirement, 
or  selected  commercial  items,  or  items  of  other  Services,  Government 
agencies  or  countries,  undergoing  military  potential  testing  (MPT). 

b.  Limited  produc'ion-urgent  (LP-U)  type.  An  item 
under  development,  available  from  other  Services,  Government  agencies 
or  countries  or  a  commercial  item  which  does  not  qualify  for  the  adopted 
category  (incomplete  test  and  evaluation)  but  which  has  been  approved 

by  HQ,  DA  for  procurement  and  distribution  in  limited  quantities  to 
meet  an  urgent  operational  requirement  that  no  adopted  item  will 
satisfy.  LP-U  is  a  tentative  type  classification  valid  for  specific 
geographic  areas  and/or  distribution,  and  for  specific  periods  of  time,- 
not  to  exceed  15  months  of  operational  use.  Items  proposed  f or  LP-U 
must  meet  the  lollowing  criteria: 

(1)  The  requirement  for  the  item  must  be 
validated  by  HQ,  DA  (ACSFOR). 

(2)  Item  must  satisfy  the  requirement  and 
involve  no  more  than  a  moderate  technical  risk. 

(3)  Item  can  be  maintained  and  logistically 
supported  in  the  geographic  area  and  for  the  time  frame  for  which  the 
classification  is  proposed,  LP-U  is  also  applicable  to  certain  high 
dollar/high  density  items  procured  with  PEMA  funds  for  evaluation 
under  the  ENSURE  Program  (AR  71-1)  as  determined  on  a  case-by-case 
basis. 


c.  Limited  production-test  (LP-T)  type.  High 
dollar  cost  or  other  selected  major  end  items,  which  have  successfully 
completed  development  acceptance  testing  (DAT),  but  successful  com¬ 
pletion  of  production  acceptance  testing  (PAT)  is  required  prior  to 
adoption.  Items  which  have  undergone  contract  definition  will  be  con¬ 
sidered  for  LP-T  classification  prior  to  subsequent  type  classification 
actions.  This  classification  authorizes  the  procurement  of  production 
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models  in  limited  quantities  for  the  purpose  of  conducting  production 
acceptance  testing  (PAT)  and  other  user  field  tests,  experiments,  and 
evaluations  of  the  item  required  to  support  subsequent  decisions  and 
activities. 


2.  Adopted  category.  This  category  applies  to  items  that 
have  qualified  for  adoption  as  follows: 

a.  Standard-A  (STD-A)  type.  A  preferred  and  fully 
acceptable  item  which  has  successfully  completed  all  required  test 
and  evaluation,  meets  DA-approved  requirements,  military  charac¬ 
teristics,  and  specifications  for  worldwide  or  specified  geographic 
areas,  is  totally  suitable  for  performing  the  required  mission,  can 
be  properly  maintained  and  logistically  supported  in  the  area  or 
environment  in  which  item  is  to  be  used,  and  is  being,  or  can  be 
produced  in  quantity. 

b.  Standard-B  (STD-3)  type.  A.n  item,  v.-hich  fulfills 
a  DA-approved  military  requirement  and  is  acceptable  for  the  stated 
use,  but  is  no!  the  preferred  item  to  fulfill  the  requirement.  This 
classification  includes  those  items  previously  STD-A  and  being  replaced 
by  a  new  item,  and  items  for  which  STD-B  is  in  the  initial  adopted 
designation  because  the  item  did  not  meet  all  of  the  qualifications 

for  STD-A. 


3.  Phase-out  category.  This  category  includes  those  items 
that  are  approaching  obsolescence  and  ultimate  disposal.  The  two  phase¬ 
out  designations  are: 

a.  Contingency  and  training  (C&T)  type.  An  item  not 
acceptable  for  U.  S.  Army  operational  requirements,  but  being  retained 
to  meet  contingency  requirements  pending  availability  of  STD-A  or 
STD-B  items,  or  training  requirements. 

b.  Obsolete  (OBS)  type.  An  item  no  longer  required 
or  acceptable  for  U.  S.  Army  use  and  to  be  withdrawn  from  troop 
use  and  disposed  of  in  accordance  with  appropriate  regulations. 

Test  and  evaluation  of  materiel  during  R&D  is  a  continuous  scru-B 
of  inter-related  and  coordinated  activities  conducted  to  provide  informa¬ 
tion,  to  individuals  or  agencies  responsible  for  decision  during  the 


development  cycle.  In  general,  materiel  testing  provides  physical 
measurement  against  prescribed  technical  specifications  and  performance 
standards,  while  evaluation  includes  judgmental  assessments  in  qualita¬ 
tive  aspects  of  military  worth  and  suitability  considering  the  threat, 
doctrine,  organization,  operational  employment,  maintainability, 
reliability  and  other  pertinent  factors.  Effective  test  and  evaluation 
activities  are  wholly  dependent  upon  continuous  and  vigorous  participa¬ 
tion  aed  interaction  at  all  levels  of  the  developer,  user,  trainer  and 
log'jtician  agencies. 

A  key  test  planning  document  which  is  receiving  major  emphasis 
within  the  Army  and  is  now  required  for  each  development  program 
which  will  enter  Engineering  Development  is  the  Coordinated  Test 
Program  (CTP).  Preparation  of  the  CTP  is  the  responsibility  of  the 
developer,  but  the  user,  trainer  and  logistician  arc  required  to 
participate  in  the  development  of  the  CTP.  * 

One  of  the  unique  features  of  Army  organization  for  OT4.-E  is  the 
interaction  which  occurs  as  a  result  of  the  Center  Team  Concept  at  the 
various  Combat  Arms  Centers.  The  Service  Test  Boards  are  collocated 
with  their  counterparts  om  CDC  and  CONARC.  For  example,  the 
Armor  and  Engineer  Board  is  located  at  Fort  Knox  close  to  the  CDC 
Armor  Agency  ar.d  CONARC  Armor  School. 

The  Service  Test  Boards  are  primarily  concerned  wi»h  suitability 
of  the  equipment  for  issue  to  the  Field  Army.  About  half  of  their  effort 
is  spent  on  .Service  Testing  with  the  remainder  being  spent  on  Develop¬ 
mental  Suitability  Tests /Military  Tests  and  Initial  Production 

Testing. 

A  brief  description  of  several  of  the  OT&E  type  tests  performed 
on  Army  materiel  is  discussed  below  (See  Inclosure  E). 


1.  Developmental  Suitability  Test  (DST).  A  new  category  of 
OT&E,  similar  to  the  Military  Potential  Tost,  which  will  be  conducted 
during  expanded  contract  definition  to  piovide  an  early  deterrr ''nation  of 
potential  military  worth  of  a  new  system.  This  test  will  normally  be 
conducted  at  a  Service  Test  Board  and  use  typical  user  personnel  to 


operate  the  equipment. 
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2.  Military  Potential  Test  (MPT).  A  test  of  an  item, 
component  or  system  for  which  no  definitive  characteristics  have 
bsen  established,  and  which  is  conducted  under  the  provisions  of 
AR  705-5  for  the  purpose  of  determining  whether  the  materiel  or 
equipment  has  military  potential  to  satisfy  a  stated  requirement. 

The  MPT  is  normally  a  limited  ter*  conducted  under  field  conditions 
and  does  not  negate  the  requirement  for  an  Engineering  Test  (ET)  or 
Service  Test  (ST).  The  MPT  would  be  conducted  by  the  appropriate 
Service  Test  Board  or  by  the  new  Project  MASSTER  Test  organization 
at  Fort  Hood,  Texas. 

3.  Operational  Service  Test  (OST).  This  is  a  new  test 
which  will  expand  on  the  previous  Service  Test  in  an  attempt  to 
obtain  the  most  realistic  operational  evaluation  of  the  item  of  equip¬ 
ment  by  including  field  exercises  simulating  combat  operations  using 
a  small  Army  tactical  element  that  would  normally  operate  the  equip¬ 
ment.  The  test  materiel  is  operated  under  simulated  tactical  conditions 
similar  to  those  expected  in  the  areas  of  intended  operational  use. 

The  purpose  of  the  OST  is  to  determine  whether  or  not  the  materiel 
is  suitable  for  its  intended  use  by:  (1)  measuring  to  what  degree  the 
materiel  meets  performance  standards  specified  in  the  requirements 
document,  (2)  field  testing  a  small  unit  equipped  with  the  materiel 
to  form  initial  judgments  on  the  overall  item/unit  effectiveness  or 
military  worth,  and  (3)  testing  and  evaluating  the  materiel  maintenance 
package. 

The  operational  type  testing  described  above  that  is  conducted 
during  the  development  and  production  acceptance  testing  cycle 
(AR  70-10)  is  followed  closely  by  more  extensive  CT&E  using  materiel 
representative  of  the  final  production  process  under  the  overall  super¬ 
vision  of  ACSFOR  and  CDC. 

These  arc  user  field  tests,  experiments  and  evaluations  to 
establish  the  actual  performance  capabilities  of  Army  equipment  in 
the  hands  of  the  user  as  well  as  the  effectiveness  of  organizational 
concepts,  doctrine,  tactics  and  tables  of  organization. 

The  CG,  CDC  is  responsible  for  planning,  programming,  budgeting 
and  evaluating  results  of  troop  test  and  field  evaluations;  all  aspects  of 
field  experiments  normally  conducted  at  CDCDC;  developing  special 
test  instrumentation;  assisting,  monitoring,  and  observing  confirmatory 
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tests  and  combat  evaluations  conducted  by  oilier  commands;  and 
reviewing  the  results  of  all  such  tests  and  evaluations.  The  CG» 

AMC  participates  in  the  planning  and  conduct  ot  confirmatory  tests 
and  supports  C DC  in  other  user  tests  or  evaluations.  All  of  the 
field  commanders,  including  CONARC  and  Army  overseas  commanders, 
are  responsible  for  conducting  troop  tests,  field  evaluations  and  con¬ 
firmatory  tests  and  reporting  the  test  results  to  CDC.  A  brief  des¬ 
cription  of  these  tests  are  shown  below; 

1.  Troop  test.  A  troop  test  is  a  test  conducted  by  a  tactical 
unit  in  the  field  to  determine  the  overall  workability  and  effectiveness 
of  present  or  proposed  organizational  concepts,  doctrine  techniques 
and  tactics  or  to  gain  further  information  on  materiel. 

2.  Field  evaluations.  The  field  evaluation  is  conducted 
under  norma'  operating  conditions  over  an  extended  period  of  time, 
to  examine  new  or  revised  doctrine  and  organization,  or  examining 
selected  weapons  systems. 

3.  Confirmatory  test.  This  is  an  intensive  user  test 
conducted  under  field  conditic  ns  by  operational  Army  units  equipped 
with  early  production  models  of  selected  weapons  systems.  The 
purpose  of  this  test  is  to  obtain  equipment  performance  experience 
which  will  minimize  unexpected  equipment  failures  in  combat. 

4.  field  experiments.  The  field  experiment  is  a  co.'.U  u..ed 
exercise  conducted  to  collect  objective  data  on  a  specific  problem  area 
for  use  in  developing  or  evaluating  new  operational  and  organizational 
objectives,  concepts,  tactics,  techniques,  procedures,  qualitative 
materiel  development  objectives  or  qualitative  materiel  requirements. 

5.  Combat  evaluations.  These  arc  format  evaluations 
designed  to  record  experience  in  active  combat  operations  as  the 
basis  for  improving  the  effectiveness  of  forces  currently  engaged  in 
combat  and  of  the  Army  as  a  w>-ole. 

The  test  objectives  set  fV-in  for  the  Project  MASSTER  Test 
Program  at  Fort  Hoed,  Texas,  utilizing  Brigade -size  Armv  units  in 
direct  support  are  listed  below; 
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a.  Improve  the  Army's  overall  combat  capability  through 
testing  and  experimentation  in  support  of  the  development  of  an 
Integrated  Battlefield  Control  System. 

b.  Recommend  materiel  requirements  and  doctrinal, 
organizational,  and  tactical  concepts  which  will  improve  Surveillance, 
Target  Acquisition  and  Night  Observation  (STANO)  capability  of  the 
bripade  and  its  supporting  forces  bearing  in  mind  the  interface  with 
and  support  provided  by  the  division,  corps  and  field  army. 

c.  Assist  in  insuring  that  STANO  materiel  under  development 
or  offered  to  the  Army  will  meet  essential  needs  of  the  Army. 

The  emphas.s  which  the  Army  is  now  placing  on  improving  the 
overall  planning  for  and  conduct  of  operational  evaluations  and  opera¬ 
tional  testing  should  become  evident  in  our  weapons  system  develop¬ 
ment  programs  over  the  next  several  years.  Maximum  utilization 
of  test  results  and  insuring  a  close  tie  with  the  Center  Commanders 
at  our  various  combat  arms  centers  should  make  certain  that  test 
results  are  placed  m  proper  perspective  when  arriving  at  decisions 
on  future  weapons  systems. 
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SYSTEM  LIFE  CYCLE  PHASES/TYPE  CLASSIFICATION  DESIGNATIONS 


APPENDIX  C 


OPERATIONAL  TEST  AND  EVALUATION 


IN  THE  NAVY  AND  MARINE  CORPS 


NAVY  OPERATIONAL  TEST  AND  EVALUATION 


The  organization  and  methods  used  by  the  Operational  Test  and 
Evaluation  Force  most  clearly  define  the  Navy's  method  of  conducting 
operational  tests  and  evaluation. 

A  few  Navy  weapon  systems  have  been  operationally  tested  and 
evaluated  through  a  different  chain  of  command;  for  example,  the 
Polaris  missile  system.  For  the  purpose  of  this  paper,  however, 
the  Operational  Test  and  Evaluation  Force  is  considered  to  be  the 
Navy's  operation.  1  testing  organization. 

A  glossary  of  terms  which  may  be  unfamiliar  to  the  reader  is 
included  at  the  end  of  this  section  of  Appendix  C. 


COMOPTEVFOR  ORGANIZATION 


The  mission  of  COMOPTEVFOR,  is  unique  among  Navy  commands 
The  organization  of  OPTEVFOR  is  also  unique.  COMOPTEVFOR  is 
a  Commander  in  the  operating  forces  of  the  Navy  with  opera-' 
tional  responsibilities  to  CNO  and  the  Commanders-in-Chief 
(CINC’s)  of  both  Fleets.  He  exercises  operational  control 
of  various  Fleet  units,  including  three  air  squadrons, 
assigned  by  the  Fleet  Commanders-in-Chief  for  the  prosecu¬ 
tion  of  RDT&E  tasks.  He  does  not  exercise  administrative 
control  of  units  assigned  for  operational  control  and  his 
organizational  position  is  therefore  not  akin  to  either  that 
of  Fleet  Type  Commanders  or  Operational  Commanders  within 
type  commands. 

The  commanders  of  OPTEVFOR  Squadrons  and  Detachments 
(TEVDETS)  are  based  ashore  with  their  staffs  and  carry  out 
RDT&E  project  activities.  These  commanders  report  to  COM¬ 
OPTEVFOR,  but  their  commands  are  loyistically  dependent  to 

/ 

a  major  degree  upon  the  base  or  facility  which  hosts  and 
supports  them. 

OPTEVFOR  is  an  independently  operating  portion  of  the 
Navy  which  i3  highly  dependent  on  outside  support.  The  Force 
is  organized  and  dispersed  to  employ  shore  facilities  on  both 
coasts  and  sea-going  resources  of  both  Fleets.  In  carrying 
out  its  tasks,  close  liaison  and  coordination  are  maintained 
with  elements  of  the  Naval  Material  Command,  System  Commands, 
of  the  Navy  Department.  This  Section  describes  the  organ¬ 
ization  of  the  Force  and  its  elements,  and  the  relationships 
with  other  Navy  commands  and  support  activities. 


COMMAND  RELATIONSHIP 

In  accordance  with  OPNAV  5440. 47B,  COMOPTEVFOR  is 
under  the  command  of: 

a.  Commander-in-Chief,  Atlantic  Fleet 

for  operational  matters  under  purview 
of  CINCLANTFLT. ■*  '  ' 

b.  Commander-in-Chief,  Pacific  Fleet  • 
for  operational  matters  under  purview 
of  CINCPAC FLT. 

c.  The  Chief  of  Naval  Operations  for 
technical  direction  and  general  policy 
guidance  for  all  matters  relating  to 
the  overall  Navy  RDT&E  program. 

These  relationships  are  shown  in  Figure  1. 


1.  Technical  direction  &  general  policy  guidance 

2.  Command,  operational  matters.  Logistic  services 

3.  Direct  access  for  technical  matters 

4.  Direct  liaison  for  technical  matters  pertaining 
to  RDT&E 

5.  Administrative  Control 


FIGURE  1. 


OPTEVFOR  COMMAND  RELATIONSHIPS 
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The  command  organization  of  OPTEVFOR  is  shovm  in  Figure  2. 

The  Deputy  Commander  at  San  Diego,  Commanders  of  the 
three  Test  and  Evaluation  Detachments,  and  the  Commanding 
Officers  of  the  three  Air  Test  and  Evaluation  (VX)  Squadrons 
are  under  the  operational  control  of  COMOPTEVFOR.  The 
Commander  also  exercises  administrative  command  of  the  Detach¬ 
ments. 

The  VX  squadrons  are  technically  squadrons  of  the  Navy 
Operating  Forces.  VX-1  ig  under  administrative  control  of 
Commander  Naval  Air  Force  Atlantic  and  Commander  Fleet  Air, 

Key  West;  VX-4  and  VX-5  are  under  administrative  control  of 
Commander  Naval  Air  Force  Pacific,  and  Commander  Fleet  Air, 
Miramar  and  Alameda,  respectively.  In  the  hear  future  VX-5 
will  come  under  the  administrative  control  of  Commander 


Fleet  Air  LeMoore. 
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OPTEVFOR  ORGANIZATION 


. ,,  Commanding  Officer 
VX  •  5 
Cnlna  lake 


n 


z 


r 


Ihe  manning  level  for  the  Operational  Test  and  Evaluation  Force 
is  275  officers  and  1,000  enlisted  tr.en.  This  includes  the  headquarters 
command  ar.d  all  subordinate,  rem-itely  located  units. 

Fleet  Development  Groups 

Fleet  Development  Groups  are  employed  by  Fleet  Commanders 
to  develop  Fleet  ideas  for  new  developments  to  determine 
feasibility.  COMOPTEVFOR  is  charged  by  OPNAV  5440. 47B  tc 
supervise  the  prosecution  of  CNO  approved  RDT&E  projects 
assigned  to  Fleet  Development  Groups. 

In  such  cases,  Commanders  of  Fleet  Development  Groups 
report,  when  directed  by  the  appropriate  Fleet  Commander-in- 
Chief,  to  COMOPTEVFOR  for  additional  duty  in  connection  w.'th 
the  projects  so  assigned.  Presently  established  Fleet  Deve¬ 
lopment  Groups  ares 
-Atlantic  Fleet 

Submarine  Development  Group  Two 

Destroyer  Development  Group  Two 
Pacific  Fleet 

Submarine  Development  Group  One 

Destroyer  Development  Group,  Pacific 
Other  Fleet  Commands 

COMOPTEVFOR  is  authorized  by  Atlantic  and  Pacific  Fleet 
Commanders-in-Chief  to  maintain  direct  liaison  with  Fleet 
Commands  and  units  in  connection  with  RDT&E’ pro  jectr, .  DEP- 
C0M0PTE,,p0R  has  similar  authority  for  Pacific  projects, 

Wavy  Shore  Establishment 

COMOPTEVFOR  is  authorized  direct  liaison  with  "Chiefs  of  Deve¬ 
loping  Bureaus  or  Offices"  for  all  technical  matters  relating 
to  the  Navy  RDT&E  program. 
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NAVY  STAFF  ORGANIZATION  POt.ICY 

Fundamental  guidance  for  the  organization  of  a  Navy  staff 
is  provided  in  Article  0508  of  Navy  Regulations, 

"The  Staff  shall  be  organized  into' such  divisions 
as  may  be  prescribed  by  the  Commander  or  by 
higher  authority.  These  divisions  shall  conform 
in  nature  and  name,  as  practical  and  appropriate, 
to  those  of  the  staff  of  .seniors  in  the  chain  of 
Command". 

The  specialized  nature  of  OPTEVFOR  functions  precludes  effective 
organization  along  the  lines  of  Fleet  or  Type  Commander  staffs. 
The  end  product  of  Fleet  and  Type  Commanders  is  Naval  opera¬ 
tions,  while  the  primary  end  products  of  COMOPTEVFOR  are  reports 
and  services. 

m 

THE  HEADQUARTERS  STAFF  ORGANIZATION 

In  general,  the  Headquarters  Staff  is  organized  by  warfare 
areas,  as  shown  in  figure  -3 ,  and  resembles  the  organization 
of  the  Staff  of  the  Deputy  Chief  of  Naval  Operations  (Develop¬ 
ment),  (DCNO(D) ) . 

COMOPTEVFOR  has  liaison  with  the  Office  of  DCNO(D) .  Respon¬ 
sibilities  and  Duties  of  the  Deputy  Chief  of  Staff  for  Opera¬ 
tions  include: 

"Maintain  liaison  with  the  offiae  of  the  DCNO(D) 
(OP-07)  and  with  other  commands  and  activities 
as  necessary,  to  carry  out  assigned  duties". 

Each  of  the  other  Staff  Division  Directors  is  directed 
to: 

"Maintain  liaison  with  developing  agencies....". 


PROJECT  COORDINATION 


OPNAVINST  3930.8  assigns  to  < the  Development  Planning 
Division  (OP-07)  within  the  Staff  of  the  DCNO(D)  (OP-07).  the 
responsibility  to  “coordinate  and  promulgate  all  OPNAV  directives 
to  COMOPTEVFOR  pertaining  to  test,  evaluation  and  investigation 
of  new  developments  by  the  Operating  Forces  of  the  Navy". 

OPTEVFOR  COMMANDS  ORGANIZATION 

Deputy  COKOPTEVFORPAC  Organization 

The  Pacific  Staff  (Figure  4  )  is  organized  on  a  near 

parallel  with,  and  is  considered  an  extension  of,  the  Nor¬ 
folk  Headguarters  staff.  The  Charter  of  the  Commander  includes 
supervision  of  VX-4  and  VX-5  activities. 


(1)  Supervise  and  coordinate 

FIGURE  4.  DEPUTY  COMOPTEVFORPAC  HEADQUARTERS  STAFF 
ORGANIZATION 
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OPNAVINST  3930.8  assigns  to>the  Development  Planning 
Division  (OP-07)  within  the  Staff  of  the  DCNO(D)  (OP-07).  the 
responsibility  to  “coordinate  and  promulgate  all  OPNAV  directives 
to  COMOPTEVFOR  pertaining  to  test,  evaluation  and  investigation 
of  new  developments  by  the  Operating  Forces  of  the  Navy". 

OPTEVFOR  COMMANDS  ORGANIZATION 

Deputy  COMOPTEV FORPAC  Organization 

The  Pacific  Staff  (Figure  4  )  is  organized  on  a  near 

parallel  with,  and  is  considered  an  extension  of,  the  Nor¬ 
folk  Headquarters  staff.  The  Charter  of  the  Commander  includes 
supervision  of  VX-4  and  VX-5  activities. 


FIGURE  4. 
ORGANIZATION 


(1)  Supervise  and  coordinate 
DEPUTY  COMOPTEVFOR PAC  HEADQUARTERS  STAFF 
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Key  West  TEVDET  Organization 
Since  its  establishment  the  Key  West  TEVDET  h»s  been 
o’riented  toward  Underseas  Warfare.  Consequently,  the  Key  West 
Staff  (Figure  6  )  is  organized  for  and  emphasizes  the  prose¬ 

cution  of  underseas  warfare  projects,  and  primary  coordination 
is  with  the  40  Division  at  Norfolk  Headquarters.  The  Analysis 
Division  at  KWTD  is  unique  among  the  detachments  and  squadrons 
of  the  Force  as  the  manpower  available  in  the  division,  with 
assistance  from  the  OEG  Representative  and  contractor  personnel, 
is  approximately  equivalent  to  the  total  number  of  analysis 
personnel  throughout  the  remainder  of  the  Force.  • 


FIGURE  6.  COMMANDER  KEY  WEST  TEVDET  STAFF  ORGANIZATION 

C-10 


Norfolk  TEVDET  Relationships.  The  external  relationship* 
of  the  Commander,  Norfolk  TEVDET,  are  shown  in  Figure  9. 

Scneduling  of  services  is  accomplished  through  the  Head¬ 
quarters  Operations  Division. 


(1)  Operational  &  Administrative  Command 

(2)  Administration  of  Norfolk  TEVDET  assigned  enlisted 
personnel. 

(3)  Communications  guard  for  Norfolk  TEVDET  by  COMOP- 
TEVFOR 

(4)  Logistic  Support 

(5)  Aviation  Support 

(t>)  OPCON  and/or  scheduling  control 
(7)  Project  Coordination 


FI CURE  9.  COMMANDER  NORFOLK  TEVDET  EXTERNAL  RELATIONSHIPS 
New  London  TEVDET  is  staffed  and  organized  to  provide 
services  to  assist  various  developing  agencies  in  the  sea  phase 
of  equipment  development  for  surface,  submarine  and  airborne 
anti-subrnsrine  warfare. 
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There  are  three  officers  on  the  Commander 's  Staff  and  four 
enlisted  personnel  are  assigned.  The  Staff  organization  ia 
shown  in  Figure  10. 


FIGURE  10..  COMMANDER  NEW  LONDON  TEVDET  STAFF  ORGANIZATION 

New  London  TEVDET  Relationships.  The  principal  relation- 

3  S 

ships  of  Commander,  New  London  TEVDET,  are  v/ith  the  Headquarters 
U,  S.  Navy  Underwater  Sound  Laboratory  (USN/lfeL),  destroyer  and 
submarine  operational  commanders  in  the  New  England  area,  and 
various  development  agencies. 


These  relationships  are  diagramed  in  Figure  11.  New 
London  TEVDET  occupies  office  space  in  a  USN/USL  building. 


(1)  Operational  and  Administrative  Command 

(2)  Support 

(3)  Project  Coordination  , 

(4)  Communications  Support  (message) 

(5)  Provision  cf  services 

(6)  OPCON  anci/or  schedule  control 


FIGURE  11.  COMMANDER  NEW  LONDON  TEVDET  EXTERNAL 
RELATIONSHIPS 
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Air  Test  and  Evaluation  Squadron  One  (VX-1)  Organization 


The  VX-1  Squadron,  Figure  below, is  organized  to  be  res¬ 
ponsive  to  the  test  and  evaluation  requirements  of  airborne 
ASW  systems.  The  Analysis  Division  at  VX-1  is  staffed  by 
military  analysts,  v/ith  assistance  from  an  OEG  Representative. 
The  squadron  is  organized  along  a  project-line  structure  to 
support  the  P-3C  evaluation. 


DIRECTOR 
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-  —  ...  -  j  (JLG  J 

PROJECTS  PROSECUTION 
DEPARTMENT 


ANALYSIS 

TECHNICAL  | 

PROJ.  TEAM 

DIFAR 

P-3C 

PROJ  TEAM 

SECTION 

SECTION  I 

LEADER 

PROJ. 

PROJ. 

LFA  DER 

TEAM 

TEAM 

VX-1  Rel iticnships.  VX-1  is  located  at  NAS  Boca  Chica. 
The  local  trea  Commander,  Naval  Base,  Key  West,  exercises 
military  command  of  the  Naval  Station  and  Naval  Air  Station. 

He  is  also  Commander,  Tleet  Air,  Key  West.  The  relationships 
of  Commanding  Officer,  VX-1,  are  shown  in  Figure  12. 

Scheduling  of  ship  and  submarine  services  is  accomplished 
through  COMOFTEVFOR  representation  at  the  quarterly  CINCLANTFLT 
scheduling  conferences. 


(1)  Command 

(2)  Command,  Operational  &  Technical  Contiol 

(3)  Local  area  military  command 

(4)  Administrative 

(5)  Support 

(6)  Project  liaison 


FIGURE  12.  COMMANDING  OFFICER  VX-1  RELATIONSHIPS 
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Air  Test  and  r  '  .luation  Squadron  Four  (VX-4)  Organization 
VX-4  is  engaged  in  test  and  evaluation  projects  for 
fighter  aircraft  and  fighter  weapons  systems.  The  squadron  is 
organized  to  emphasize  project  operations  as  shown  in  Figure  13. 


FIGURE  13.  VX-4  SQUADRON  ORGANIZATION 

V>:  -4  Relationships .  VX-4  is  an  opercitional  squadron  of 
the  Pacific  Fleet,  and  the  Commanding  Officer,  as  senior  Fleet 
Naval  Aviator  of  a  Fleet  air  unit  based  at  NAS,  Point  Mugu,  has 
additional  duties  as  Comma. .der,  Fleet  Air  Detachment,  Flint 
Mugu  (CFAD) , 
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The  relationships  of  Commanding  Officer*  VX-4,  are  shown  in 
Figure  14. 

The  instrumented  ranges  at  the  Pacific  Missile  Range  (PMR) 
and  Naval  Weapons  Center  (NWC),  China  Lahe,  are  used  tc  support 
fighter  aircraft  projects.  Guided  Missile  Unit  (GM»)  41  is 
collocated  with  VX-4  and  supports  missile  programs. 
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(?)  Project  Liaison 

(8)  Information 


FIGURE  14.  COMMANDING  OFFICER  VX-4  RELATIONSHIPS 
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Vi  A 


VX-5  is  engaged  in  test  end  evaluation  projects  for  Navy 
attack  aircraft  and  attack  weapon  systems.  The  squadron  is 
orqani-ed  with  emphasis  on  projecc  operations  as  shown  in 
Figure  15.  A  VX-S  Detachment  is  located  at  NAS  Oceana  con¬ 
ducting  Op  Eval  of  the  A-6  aircraft  and  associated  systems. 


COMMANDING  OFFICE* 


SAFETY  ] 
1  OFFICE*  | 


FIGURE  15. 


VX-5  SQUADRON  ORGANIZATION 
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VX-5  Relationships.  VX-5  is  an  operational  squadron  of 
the  Pacific  Fleet  presently  under  administrative  command  of 
COMFAIR  ALAMEDA.  In  the  near  future  VX-5  will  come  under  tne 
administrative  control  of  Commander  Fleet  Air  LeMoore. 

As  senior  Naval  Aviator  of  a  Fleet  air  unit  based  at  NWC,  China 
Lake,  the  Commanding  Officer  of  VX-5  has  additional  duties  as 
CFAD,  China  Lake.  The  relationships  are  shown  in  Figure  16. 

The  squadron  is  the  principal  user  of  NVC  China  Lake  "Charlie" 
and  "Echo"  Ranges. 
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FIGURE  16.  COMMANDING  OFFICER  VX-5  RELATIONSHIPS 
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FUNDING 

Under  present  Navy  funding  procedures,  the  Force  is 
heavily  dependent  or  funding  support  by  other  Navy  commands  and 
activities.  The  total  financial  resources  directly  available 
to  the  Commander  for  allocation  are  approximately  1.2  million 
dollars  per  year,  which  is  less  than  five  percent  of  the  total 
cost  of  carrying  out  the  tasks  assigned.  Of  the  1.2  million 
dollars  made  available  yearly,  about  58%  (the  RDT&E.N  Allotment) 
is  basically  level-funded  through  the  Naval  Ships  Systems 
Command  (NSSC)  and  no  budgeting  by  COMOPTEVFOR  is  required. 

An  a«rv.al  budget  must  be  submitted  for  O&M,  N  funds  (about 
$500K)  for  Force  use. 

The  financial  staff  at  COMOPTEVFOR  Headquarters  consists 
of  one  Supply  Corps  officer.  The  only  other  Supply  Corp3 
officer  in  the  Force  is  at  Key  West  TEVDET,  the  allowance  for 
Supply  Officers  in  the  VX  squadrons  having  been  rescinded 
within  recent  years.  The  Force  is  not  staffed  for  major 
budgeting  and  accounting  functions.  Accounting  tasks  are  per¬ 
formed  for  OFTEVPOR  by  various  Fleet  and  shore  activities. 

COMOPTEVFOR  presently  is  supported  by  financial  resources 
from  several  appropriations  with  examples  as  follows: 

RDT&E, N 

1.  Direct  allotment  (about  $7C0K)  annually  via  NAV- 
SHIPS'fSCOM  for  minor  instrumentation  and  travel 
connected  with  projects. 
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2. 


Program  VI  funding  which  supports  activities 
engaged  in  RDT&E,  such  as  NWC,  China  Lake  and 
NMC  Pt.  Mugu. 

3.  Funds  which  accompany  some  projects  into  the  T&E 
phase  and  wh:.ch  have  been  programmed  and  budgeted 
,for  that  purpose. 

Q&M,  N 

1.  Direct  allotment  annually  from  the  r PNAV  Comp¬ 
troller  to  COMOPTEVFOR  (about  $270K)  for  general 
support  of  OPTEVFOR  Headquarters,  Norfolk  TEVDET. 
This  includes  utilities,  operations,  maintenance, 
and  minor  construction  at  Norfolk  and  Norfolk  TEVDET. 

2.  Fleet  operating  and  maintenance  funds  for  ships 
and  aircraft  including  the  VX  squadrons,  and  for 
buying  range  time. 

3.  Direct  allotment  from  OPNAV  Comptroller  (about  $44K 
annually)  to  DEPCOMOPTEVFORPAC,  and  $200  for  KWTD. 

4.  Support  from  shore  establishment. 

1.  Military  Personnel  costs. 

PAM,  N 

1.  Aircraft  and  spare  parts  for  VX  squadrons.' 

2.  Missiles  for  evaluation  or  for  testing  other  systems. 

MILCON 
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New  building  for  Key  West  TEVDET. 
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No  direct  identity  is  presently  established  with  the  SCN 
Appropriation. 

Because -%f  the  wide  distribution  of  funding  support,  it 
is  not  possible  to  develop  a  significantly  accurate  estimate 
of  annual  Navy  expenditures  for  OPTEVFOR  operations.  Base 
support  costs,  for  example,  are  not  estimated  by  the  host 
commands.  An  exception  is  at  NWC,  China  Lake,  where  an  esti¬ 
mate  had  been  made  of  the  support  provided  to  VX-5,  generally 
'■>ased  on  a  prorated  share  of  personnel  on 

board,  aircraft  assigned,  and  similr.r  factors;  but  many  base 
services  such  as  fire  protection,  security,  mail  and  supply  are 
not  considered.  •  Another  cost  of  Force  operations  for  which 
there  is  no  accurate  measurement  i3  the  operating  cost  for 
ships  supporting  T&E  projects.  A  few  Fleet  ships  are  funded 
by  the  RDT*E  Appropriation,  but  no  estimate  is  made  of  the 
percentaa'j  of  their  costs  which  could  be  attributed  to  the 
operations  of  OPTEVFOR. 


PONDING  OF  PROJECTS 

Source  and  Uses  of  COMOPTEVFOR  Funds.  COMOPTEVFOR 
receives  an  annual  allotment  of  Research  and  Development,  Test 
and  Evaluation  funds  from  NAVSHIPSYSCOM  as  CNO’s  agent,  to  sup¬ 
port  its  project  work,  Tho  major  portion  of  these  funds  aro 
used  for  travel  and  per  diem  expenses  of  Project  supervisors 
and  Project  Officers.  Another  large  portion  is  ueed  to  procure 
general  purpose,  relatively  inexpensive  Instrumentation  equip¬ 
ment  as  recommended  and  approved  by  the  Instrumentation 
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Committee.  Limited  funds  are  available  to  Project  officers 
for  the  following  miscellaneous  expenses: 

a.  Spare  and  repair  parts  for  repair  and  maintenance  of 
equipment*  (electron  tubes,  adapters,  batteries,  connectors, 
switches,  capacitors,  motors,  bushings,  gaskets,  bearings, 
cables,  plugs,  etc.). 

b.  Supplies  and  materials  determined  essential  for  the 
efficient  prosecution  of  the  project,  (recording  paper,  armored 
cable,  steel  plates  and  angles,  fenders,  instruments,  lumber, 
hawsers,  wire  rope,  oxygen  and  acetylene  gas,  plastics,  paint, 
electrical  wire,  hardw~rd,  etc,). 

c.  Services  of  Public  Works  Departments  and  shipyards  for 
minor  installation,  repair  and  maintenance  of  equipments, 
installation  of  power  lines  between  power  sources  and  evaluation 
equipments,  etc.  Prom  time  to  time  other  commands,  activities, 
systems  commands  and  Developing  Agencies  oj:  offices  give  COM- 
OPTEVFOR  or  its  detachments  additional  funds  to  prosecute 
particular  projects.  Such  fundings  is  arranged  for  by  the 
Project  Officer. 

Other  Support  and  Assistance.  As  tactics,  systems  and 
equipments  become  more  complex  and  sophisticated,  COMOPTEVFOR 
depends  more  upon  the  support  and  assistance  of  the  Developing 
Agencies  and  System  Commands. 

a.  OPNAV  INSTRUCTIOU  3960.1  (serioo)  prescribes  that  the 
Developing  Agency  will  assist  COMOPTEVFOR  in  the  preparation  of 
the  technical  phase  portion  of  the  project  plan,  will  furnish 
required  material  and  technical  support,  including  spare  parts 


and  special  test  equipment,  and  will  make  necessary  arrangements 
for  furnishing  the  installation  and  removal  of  equipment. 

b.  The  project  assignment  letter  or  message  from  CNO 
often  tasks  a  Systems  Command,  CNM  or  a  particular  activity 
such  as  NWC  China  Lake  or  NAVMISCEN  to  provide  support  for  a 
specific  project. 

c.  Other  examples  of  support  are: 

(1)  Target  and  aircraft  tracking  services  from 
weapons  ranges. 

(2)  Telemetry  readout  from  FMSAEG. 

(3)  Target,  aircraft  tracking  and  telemetry  services  . 
from  NMC/PMR. 

•  * 

(4)  Project  instrumentation  and  data  analysis 
services  from  APL/ Johns  Hopkins  University. 

(5)  Missile  preparation  and  certification  for  tests 
by  NWS  Yorktown,  Charleston,  Seal  Beach  and  Concord. 

(6)  Range  services  and  tracking  data  from  ETR  Cape 
Kennedy  and  NOU  Patrick  AFII. 

(7)  Ship  system  checkout,  trouble  shooting,  personnel 
training  for  project  ships,  and  installation  of  instrumentation 
from  NAVSHIP8  ENG5TA  Port  Huoneme. 


(9)  Ship  end  aircraft  services  from  the  TYCOMS  and 
numbered  fleets  (scheuulod  by  Operations  Division). 

(10)  RAV  and  TA-,  for  ship  checkout  and  instrumen¬ 
tation  installation  from  the  TYCOMS  and  numbered  fleets. 

(11)  Scheduling  of  services  by  the  JAX  and  VACATES 
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area  coordinators. 

(12)  Instrumentation  and  data  analyt>is  fret-  NOTS 
China  Lake. 

(13)  Photographic  services  from  Camera  Group, 

Atlantic  and  Pacific  and  the  Naval  Photo  Center  Anacostia. 

(14)  Computer  services  from  the  LANTFLT  OPCON  Center 
and  Fort  Eustis  (Army) . 

(15)  Procurement  and  installation  of  equipment  to 
be  evaluated  from  the  developing  SYSCOH,  agency  or  office. 

(16)  Data  acquisition,  analysis  and  reduction 
services'  from  the  developing  SYSCOM,  agency  or  office. 

(17)  Training  of  personnel  by  fleet  schools. 

(18)  Test  ranges  for  project  operations  by  AUTEC. 
AFWR,  NOLTF  Fort  Lauderdale. 

(19)  Computer  support  from  NOTS  Pasadena. 

Funding  of  Ships  Under  the  Operational  Control  of  COMOP- 

TEVFOR 

a.  Supplies,  equipage  and  maintenance  are  funded  by  the 
Type  Commander. 

b.  When  COMOPTEVFOR  is  responsible  for  evaluation  of 
a  major  equipment  such  as  a  complete  missile  system  or  a  new 
radar,  the  sponsoring  Systems  Command  or  Developing  Agency  is 
responsible  for  financing  the  installs. ion  of  the  equipment 
and  the  costs  of  replacement  parts  and  components  required  to 
keep  the  equipment  operational. 

c.  COMOPTEVFOR  is  responsible  for  the  procurement  of 

the  tools  of  the  evaluation  ouch  as  test  equipment,  photographic 
supplies,  magnetic  tape,  and  miscellaneous  consumable  hardware, 
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|  Responsibilities  of  Project  Officer. 

'  a.  From  the  information  available  in  the  TOP,  project 

I 

assignment  letter  and  liaison  with  the  Developing  Agency, 
determine  and  itemize  the  estimated  funding  requirements  in: 

(1)  personnel 

(2)  material 

(3)  services 

(4)  schools 

(5)  travel  and  per  diem 

b.  Discuss  with  the  cognizant  Developing  Agency  and  the 
Force  comptroller  the  funding  of  the  above  items. 

c.  Obtain  the  assistance  of  the  Comptroller  in  costing 
requirements  where  necessary. 

d.  Arrange  for  support  and  special  funding  that  may  be 
required . 

COMOPTEVFOR  MISSION  AND  FUNCTJ-  'NS 

The  environment  in  which  the  Commandr  -  Operational  Test 
and  Evaluation  Force  undertakes  to  fulfill  his  mission  has 
undergone  change  since  the  Force  was  chart'  red.  Department 
of  Defense  procurement  policies  and  procedures,  and  the 
RDT&E  process  itself,  have  undergone  major  revision. 

The  Department  of  the  Navy  has  experienced  major  reor¬ 
ganizations,  the  latest  occurring  in  May  196C.  The  organi¬ 
zational  and  policy  changes  have  had  an  impact  upon  OPTEVFOR. 
In  addition,  the  systems  and  subsystems  that  OPTEVFOR  tests 
and  evaluates  have  become  more  complex,  and  the  evaluations 
require  a  high  level  of  technical  and  operational  competence. 
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Complex  systems  require  lengthy  RDT&E  life  cycles  and  the 
procurement  decision  may  become  mandatory  prior  to  completion 
of  the  system's  operational  evaluation.  Consequently, 
OPTEVFOR  responsibilities  are  oi'ten  directed  toward  identi¬ 
fication  of  deficiencies  in  syster=3  In  use  and  recommenda¬ 
tions  for  improvement. 

Computer  technolog;',  simulation  and  modeling,  and  com¬ 
prehensive  data  tinalvsis  i  jtlu.iqv.os  y**  available  to 
evaluate  thoroughly  major  system  capabilities,  t.o  minimize 
usage  of  critical  Fleet  services,  and  to  reduce  the  total 
evaluation  time.  Sophisticated  resources  and  tools  such  as 
AUTEC  now  are  available  to  conduct  Navy  test  and  evaluation. 

COMOPTEVFOR  MISSION 

The  mission  of  COMOPTEVFOR  is  assigned  by  the  Chief 
of  Naval  Operations  in  OPNAV  5440. 47B  and  summarized  as 
follows: 

"Test,  investigate,  appraise,  and/or  evaluate 
specific  end  items,  systems,  tactics,  procedures, 
ai.  i  develop  tactics  and  procedures,  as  specifically 
aoeiyned.  When  directed  by  the  Chief  of  Naval 
Operations,  assist  developing  agencies," 

"The  overall  mission  is  intimately  related  to'  the 
furtherance  of  the  Navy's  RDT&E  Program." 

COMOPTEVFOR  FUNCTIONS 

The  functions  of  C0M0PTEVP0R  and  other  elements  of  the 
operating  forces  engaged  in  RDT&E  projects  are  assigned  by 
CNO  in  OPNAV  Instructions  5440. 47B  and  3960. ID. 


These  functions  are  illustrated  by  the  eight  general 
types  of  projects  which  may  be  assigned  to  COMO'PTEVFOR.  Most 
can  be  sub-classified  as  having  primarily  airborne,  V,  ship 
or  submarine,  S,  or  combined,  C,  application.  The  project 
types  and  their  designations  are  shown  in  Table 

Table  Typos  of  OPTEVFOR  Projects 


Project  Type 
Operational  Evaluations 


Designation 


1.  Ship/Sub 

0/S 

r,  1 

2.  Airborne 

o/v 

;■ 

i 

f 

3.  Combined 

0/0 

Technical  Evaluations 

T/S,  T/V, 

T/C 

)  - 

Concurrent  Evaluations 

) 

(Tech/Op) 

C/S,  C/V, 

C/C 

r *  • 

»  . 

Development  Assist^ 

* 

Fleet  Research  Tnvesti- 

D/S,  D/V, 

D/C 

!  1  < 

* ; .  * 

gations 

P/R 

■” 

Fleet  Operational 

r*  • 
t ,  • 

Investigations 

F/0 

Operational  Assists 

X/S,  X/V, 

s/c 

n 

S  ' 

Operational  Appraisals 

P/S,  P/V, 

P/C 

To  demonstrate  the  functions  of  OPTEVFOR,  the  work 
contents  of  these  types  of  projects  are  briefly  summarized 
below. 

Operational  Evaluations  (Or  Eval) .  The  test  and 
analysis  of  a  system,  component,  or  equipment  under  service 
operating  conditions,  to  determine  ability  to  meet  spec*  *..*  ?d 
operational  performance  requirements  and  design  specifications 
and  to  establish  suitability  for  service  usj. 
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A  secondary  purpose  may  be  determination  of  tactics. 

Technical  Evaluations  (Tech  Eval) .  The  tes .  and  analysis 
required  by  a  Developing  agency  (DA)  to  determine  whether 
a  weapons  system,  support  system  component  equipment,  or 
material  meets  design  specifications,  is  functioning  in  a 
technically  acceptable  manner  in  its  operational  environment,  • 
and  is  technically  suitable  for  an  Operational  Evaluation. 

Concurrent  Evaluations.  The  combination  of  a  Tech  Eval 
and  Op  Eval  into  a  single  project,  assigned  only  for  systems 
or  components  of  such  size  and  complexity  that  significant 
savings  in  time  and  resources  will  result.  A  concurrent 
evaluation  is  not  the  simultaneous  conduct  of  these  eval¬ 
uations,  but  includes  simultaneous  prosecution  of  those  testa 
or  other  project  efforts  which  are  common  to  each. 

Development  Assists.  Fleet  support  to  developing 
agencies  (DA‘s)  for  tests  needed  to  assist  in  development 
of  a  system  or  equipment  or  for  gathering  of  data  needed 
to  determine  the  direction  in  which  development  should 
proceed.  These  tests  also  may  relate  to  material  improve¬ 
ments  of  equipment  already  in  the  Fleet. 

-Fleet  Research  Investigations .  Examination  of  natural 
or  special  phenomena  in  an  operational  environment  required 
by  a  developing ' agency  in  prosecution  of  research  and  for 
Which  the  assistance  of  the  operating' forces  is  needed. 

Fleet  Operational  It'ivestigations .  Development,  exam¬ 
ination,  or  comparison  by  the  operating  forces  of  tactical 
concept?,  operating  procedures,  or  techniques.  These 
investigations  usually  involve  optimizing  use  of  the 
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equipment  and  systems  already  in  the  Fleet. 

Ooerar.ional  Assists.  The  gathering  of  performance 
data  on  a  subsystem  or  component  prior  to  either  a  Tech 
Eval  or  Op  Eval.  These  tests  provide  essentially  a  "quick 
look"  evaluation  in  those  cases  in  vhich  decisions  must 
be  made  as  to  the  worth  of  a  course  of  development,  usually- 
in  modifications  to  exif»tirg  equipment. 

Operational  Aopraisaj s.  The  appraisal  of  systems, 
subsystems,  or  components  installed  and/or  operated  in 
Fleet  units  without  an  Op  Eval. 

The  functions  assigned  to  COMOPTEVFOR  in  each  of  the 
above  project  types  are  summarized  in  Table  /. 
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COMOPTEVFOR  FUNCTIONS  RENTED  TO  TOE  RDT&E  CYCLE 
The  tine-phased  relationship  between  the  RDT&E  cycle  and 
the  assignment  of  projects  to  COMOPTEVFOR  is  indicated  in 
Figure  //.  The  Force  is  ir.\  ->lsed,  in  varying  degrees, 
with  all  categories  of  RDT&E  from  basic  research  to  evaluation 
of  newly  developed  equipment  and  appraisal  of  systems  in 
service  use.  This  involvement  in  the  early  categories  of 
ROT&E  Research,  Exploratory  and  Advanced  Development  is 
ordinarily  on  trie  basis  of  requirements  for  Fleet  service 
assistance.  Fleet  introduction  of  the  material  being  tested 
normally  is  several  years  hence,  if  ever.  An  exception  is 
the  case  of  some  minor  hardware  items  which  routinely  go 
from  exploratory  development  to  Fleet  use.  Some  of  these 
may  be  given  to  OPTEVFOR  for  Op  Eval. 
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Equipment  which  has  progressed  into  the  category  of 
engineering  development,  or  contract  definition,  has 
imminent  Fleet  use  and  OPTBVFOR  becomes  involved  in  the 
test  and  evaluation  planning.  The  opinions  and  advice  of 
COf'^r'TEVFOR  are  inc’ »  ^er"  iv.  T\r'  S*  ‘  of  Technical 
Development  Plans  for  systems  which  he  will  ultimately 
evaluate,  or  which  are  of  such  complexity  that  a  system 
performance  model  will  be  used  in  the  development  process. 

Examination  of  the  projects  assigned  to  COMOPTEVFOR 
indicates  that  the  Force  is  less'  apt  to  be  involved  early 
in  the  development  cycle  for  aircraft  and  airborne  equip¬ 
ment,  than  for  ship  or  submarine  equipment.  Aircraft  and 
airborne  equipment  are  more  often  deployed  prior  to  Op 
Eval  than  are  ship  or  submarine  systems.  This  situation 
may  bo  attributed  to  the  availability  of  approximately 
2C0  Navy  aircraft  for  the  R&D  phases  of  projects.  OFTEVFOR 
assistance  and  involvement  in  early  phases  of  airborne  deve¬ 
lopments  arc  influenced  by  the  availability  of  the  required 
aircraft  configuration  (examples  being  the  F-4J/AWG-10) , 

The  Force  has  never  beon  requested  to  assist  in  an  airborne 
Tech  Evalj  howover,  concurrent  evaluations  have  been  assigned. 

Development  Assist  Project* 

Force  involvement  in  assist  projects  is,  in  theory, 
primarily  one  of  liaison  and  scheduling  of  Pleat  services 
in  support  of  developing  agencies.  About  60%  of  the  pro¬ 
jects  now  assigned  to  the  Force  are  development  assists. 
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Table  /.  COMOPTEVFOR  Project  Functions 


Type  of  Project 

c/'s  d/s  ?7o  f/r  o/s  r7s  x7s  p7s 

C/V  C/V  0/V  T/V  X/V  P/V 


a.  Subm! ts  comments  to  CNO  on  project 
request. 

b.  Assigns  relative  priority. 

c.  Prepares  project  plan. 


X  X  X  X  X  X  x  -x 
XXXXXXXX 
X  X  XX 


d.  Prepares  project  plan,  coordi¬ 
nating  with  the  DA  for  technical 
phase  portions.  X 


e.  Arranges  lor  and  coordinates 
Fleet  support. 


XXX 


f.  Determines  Fleet  support  require¬ 
ments  and  arranges  for  same. 

g.  Ensures  readiness  of  Fleet  units. 
Ensures  Installation  plans  have 
been  approved  by  the  DA  or  e 
designated  building  or  overhaul 
facility  for  the  Fleet  unit  on  * 
which  the  equipment  Is  installed, 

h.  Prepares  OP  orders. 


X 


X 


X  X 


X  X  X  'A 

i 

X  X  X  X 


X  X  X  X 
X  X  X  X 


9 

I.  Reports  "Project  Reediness."  X 

.  J,  Advlsr*  the  operational  commander 

on  the  projects  to  be  prosecuted,  X 


X 


X  X 


X  X 


"1 


t 

; ; 


* 


li 


n 


0 

r 


k.  Monitor*  projeer  progress  end 
reports  to  CNO  e*  directed. 

l.  A*  appropriate,  submits  recom¬ 
mendation*  to  CNO  to  terminate, 
extend,  or  modify  projects. 


X 


X  X  X  X  X  X 


X  X 
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Table  /. 


COMOPT€VfOR  Project  Functions  (Cont'd) 


Type  of  Project 


c/s  o/s  f7o~f7r“o75  17s  x/s  p/s 

C/V  0/V  O/V  T/V  X/V  P/V 


m.  Upon  completion  of  tests,  fur¬ 
nishes  report  to  CNO  of  support 
provided  and  such  other  reports 
as  specified  by  CNO. 

n.  Reports  results  to  CNO  with 
appropriate  conclusions  and 
recommendations  concerning 
suitability  for  service  use. 

o.  Collects  and  analyze*  data  and 
reports  results  to  CNO  and 
activity  requesting  the 
project. 

p.  Prosecutes  tests  with  assls-  ' 
tance  of  OA. 


XXX 


X  X 


f 

XX  XX 


7*35 


X/C,  x/s,  x/v 


Fleet  research  services  and  technical  evaluations  are 
assist  type  projects  and  inpose  about  the  same  general 
requirements  on  the  Force.  Development  assists  may  be 
assigned  (and  have  been)  for  projects  ranging  from 
exploratory  development  to  deployed  systems. 

In  the  cr>  ,e  of  airborne  development  assists,  the 
Force  often  becomes  the  provider  of  services  and  is  more 
deeply  involved  than  "liaison  and  scheduling."  VX-1  is 
assigned  a  greater  number  of  assist  requests  than  VX-4  or 
VX-5,  probably  due  to  the  scarcity  of  P-3  aircraft  outside 
operational  squadrons.  For  some  surface  assist  projects, 
the  liaison  and  scheduling  services  require  a  substantial 
amount  of  manpower  and  time  (viz.,  D/S-315,  the  Torpedo 
MK-46,  Mod  1).  In  some  cases,  the  concurrent  or  operational 
evaluation  for  the  equipment  is  imminent  and  a  good  famili¬ 
arity  base  can  be  establishr  :  thiough  Force  involvement  in 
the  assist  project. 


Operational  Evaluations 

Operational  (and  concurrent)  evaluations  are  the  prin¬ 
cipal  functions  of  the  Force  and  its  primary  reason  for 
existence.  In  the  classical  sense,  these  should  be  com¬ 
pleted  prior  to  the  equipment  being  procured  in  quantities 
and  deployed.  The  demands  for  new  and  better  systems  to 


meet  new  threats,  the  1  -  1/2  -  ?  year  lead  time  inherent 
in  planning  for  and  obtaining  appropriations,  the  time 


required  fof  tooling  in  industry  and  the  apparent  economy 
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in  early  and  quantity  procurement  commitments  all  work 
against  allowing  adequate  time  for  evaluation  prior  to 
deployment.  CNO  has  established  guidance  in  OPNAV  Instruc¬ 
tion  3960. ID  that: 

"With  increasingly  complex  systems  coming  into 
being  it  is  wore  than  ever  necessary  to  test  and 
evaluate  systems  thoroughly  in  a  wide  range  of 
operational  environments." 

In  view  of  this  guidance,  the  Op  Eval  functions  of 
OPTEVFOR  have  increased  in  scope  for  the  complex  projects. 

Fleet  Operational  Investigations 

Some  of  these  projects  equal  or  exceed  many  On  Evals 
in  urgency,  complexity,  and  demands  on  resources.  They 
usually  have  one  or  more  of  three  broad  objectives: 

1.  Determine  how  to  get  the  most  out  of  a  system 
introduced  prior  to  Op  Eval, 

2.  Determine  if  the  system 'will  operate  in  a  mode 
for  which  it  was  not  designed  nor  intended. 

3.  Determine  methods  to  use  systems  in  an  envi*- 
ronment  not  previously  considered. 

The  requirement  for  projects  of  this  type  and  the  need 
for  special  test  and  evaluation  abilities  beyond  the  operating 
forces  are  apparent  when  the  test  objectives  are  examined. 

Many  could  not  be  effectively  prosecuted  by  the  operating 
forces  as  they  are  now  organized  and  staffed. 
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Tactics 


One  of  the  missions  of  COMOPTEVFOR  is  to  •'test, 
evaluate,  appraiae,  and  develop  tactics."  The  extent  to 
which  this  function  is  performed  varies  among  elements  of 
the  Force,  in  the  tactical  warfare  area.  VX-4  and  VX-5 
are  involved  heavily  and  a  substantial  portion  of  their 
efforts  are  directed  toward  development  of  <-»ctics  and 
aircraft  tactical  manuals  (TACMANS).  Projects  such  as  F/O- 
210,  O/V-28,  and  O/V-37  are  a  fundamental  source  of  urgently 
required  tactical  information  for  Fleet  squadrons. 

In  undersea  warfare  there  is  les3  Force  involvement 
in  tactics  development.  The  submarine  forces  develop  their 
own  tactics  and  the  same  is  generally  true  of  Fleet  ASW 
groups.  VX-1  does  not  have  the  same  obligation  on  TACMANS 
for  ASW  aircraft  as  have  the  tactical  air  warfare  squadrons. 

Reports 

COMOPTEVFOR  is  required  to  "report  formally  to  CNO  the 
results  of  operational  evaluations  and  concurrent  operational 
tests  and  evaluations  with  recommendations  in  connection 
therewith,"  Reference  (a).  Reports  may  be  formally  submitted 
during  the  course  of  an  evaluation  as  Preliminary,  Progress, 
or  Partial  Reports  to  provide  information  cr.  knowledge  gained 
to  date.  A  Final  Report  is  Submitted  on  completion  of  an 
evaluation  project,  and  this  may  be  followed  by  a  Supple¬ 
mentary  Report  if  additional  and  significant  data  become 
available  after  the  Final  Report  is  submitted. 
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Commanders  of  Force  elements  may  be  directed  to  prepare 
and  distribute  Advance  Evaluation  Notes  (AEN'e)  for  material 

b 

under  evaluation.  The  purpose  the  AEN  is  to  provide  timely 
operational  training  or  maintenance  information  to  Fleet 
units  using  the  systems.  AEN's  are  promulgated  extensively 
by  the  Air  Test  and  Evaluation  Squadrons  for  aircraft  air¬ 
borne  systems  deployed  but  stili  under  evaluation. 

PROJECT  REPORTS  -  DESCRIPTION  AND  PREPARATION 

General.  CU’ICrTEVFOR  reports  are  read  by  a  wide 
eudience  with  diverse  backgrounds  and  different  areas  of 
interest.  Some  members  of  this  audience  require  an  extensive 
amount  of  technical  detail  while  others  do  not.  For  example, 
an  agency  responsible  for  the  correction  of  equipment  defi¬ 
ciencies  will  require  a  detailed  technical  description  of 
the  project  equipment.  Conversely,  a  decision  maker  or 
Fleet  user  may  require  only  a  functional  description  of  the 
equipment.  The  Project  Officer  is  charged  with  the  respon¬ 
sibility  of  writing  a  report  which  meets  a3  nearly  as  possible 
the  needs  of  all  readers. 

Objectives.  The  objectives  of  project  reports  are: 

a.  To  advise  the  Chief  of  Naval  Operations  of  the 
results  of  an  Operational  Evaluation  or  Concurrent  Evalua¬ 
tion  concerning  an  item  or  system  in  order  that  a  decision 
may  be  made  concerning  its  acceptability  Cor  service  use. 

b.  To  advise  the  Fleet  and  other  appropriate  commands 
of  the  capabilities  and  limitations  of  the  equipment  or 
nyt ;em  tested. 
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c.  To  advise  the  Chief  of  Naval  Operations,  systems 
commands  and  other  technical  agencies  of  deficiencies,  ;r<3 
to  maha  recommendations  for  improvement  to  the  equipment. 

d.  To  provide  the  Chief  of  Naval  Operations  and  other 
interested  commands  with  recommended  tactics.  These  tactics 
may  apply  to  standard  Fleet  equipment  or  systems, or  to 
newly  evaluated  devices  which  are  recommended  for  service 
use. 

e.  To  provide  the  Chief  of  Naval  Operations  and  inter¬ 
ested  commands  With  recommendations  for  design,  documenta¬ 
tion,  test  equipment/test  points,  logistic  spares  and/or 
manufacturing  techniques,  personnel  manning  and  training 
requirements,  and  other  material/human  factors  implications 
incident  to  adoption  of  a  new  equipment,  weapon  or  system. 

f.  To  record,  in  readily  accessible  form,  the  detailed 
information  which  has  been  purchased  at  extraordinary  expense 
and  for  which  needs  may  arise  in. the  futuie. 

Timeliness  of  Project  Reports 

In  the  prosecution  of  any  project  it  is  important  that 
pertinent  and  accurate  data  be  obtained  as  soon  as  practicable 
and  that  an  accurate  analysis  of  data  and  conclusions  be 
submitted  promptly  in  order  that  the  Navy  may  capitalize  on 
the  time,  effoic,  and  money  expended  in  the  prosecution  of 
the  project.  Although  accuracy  and  sound  reasoning  are  para¬ 
mount,  it  must  be  recognized  that  the  value  of  the  work  done 
deteriorates  with  delay  in  completing  the  project  report. 


Reports  to  CCMOPTEVFGR  from  Prosecutir  -•  rowmands 

a.  Routine  Periodic  Reports.  Such  reports  are  s*:b- 
fitted  to  indicate  the  status  of  active  projects  znd  to 
uummarize  progress  made  in  the  ■•erall  evaluation  -orogra:#. 

These  reports  normally  have  distribution  only  withir 
OPTFVS'OR. 

b.  SITREPS  (Situation  Reports)  or  Firing  Reports.  In 
certain  projeccs  which  are  of  priority  interest  to  COMOPTEVFOR, 
timely  reports  of  projact  progress  are  required  while  tests 
are  being  conducted  in  tne  field.  When  stipulated  in  the 
project  plan,  SITREPS  and/or  tiring  reports  will  be  sent 
"Action"  tc  the  prosecuting  command  and  "Information"  to 
COMOPTEVFOR ,  as  well  as  to  DEPCOMOPTEVFORPAC  by  commands 
prosecuting  projects  in  the  Pacific,  at  specified  intervals 
during  the  prosecution  of  a  project.  These  reports  will  be 
drafted  by  the  appropriate  on-the-ocene  prosecuting  command 

i 

representative  (normally  the  Project  Officer).  Unless 
specifically  authorized,  8ITREPS  and  firing  reports  will  not 
be  sent  to  any  activity  or  command  external  to  the  OPTEVFOR 
organization.  When  these  ’•eports  do  include  on  external 
command  as  information  or  action  addressee,  and/or  when 
originated  through  an  external  command,  the  report  will 
commence  with  the  words  "(Prosecuting  Command)  REP  SENDS." 

The  words  "OPTEVFOR  REP  SENDS"  will  be  used  only  in  those 
instance*  where  a  Staff  Headquarters  representative  is  the 
drafter. 
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When  appropriate,  such  reports  will  include  the  concurrence 
or  nor.concurrence  of  other  commands  or  parties.  Firing 
reports  and  SITPEPS  shall  be  numbered  consecutively  and  the 
final  report  shall  be  identified  bv  including  the  word 
"FINAL"  after  the  report  number  (SITF.EP  19  FINAL) .  The 
SITREP  shall  be  narrative  in  style,  whereas  the  format  of  the 
firing  report  shall  be  as  stipulated  in  the  project  plan, 
c.  Deficiency  Report 

(1)  A  deficiency  report  in  letter,  speedletter, 
or  message  form  shall  be  submitted  to  the  prosecuting  command. 
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information  to  COMOPTEVFCR  (and  DEPCOMOPTEVFORPAC  for  Pacific 
projects)  by  any  ship  or  subordinate  command  prosecuting  a 
project,  when  -the  project  is  being  delayed  because  of  its 
manifest  unsuitability  for  evaluation,  lack  of  required 
support,  or  prolonged  delay  in  delivery  of  equipment. 
Deficiency  reports  are  also  desired  on  equipments  exhibiting 
design  deficiencies  even  though  such  deficiencies  may  not 
delay  projecc  operations.  When  the  pros2Cuting  command's 
Project  Officer  is  embarked,  he  will  submit  the  deficiency 
report  and  will  preface  the  message,  "(Prosecuting  Command) 

REP  SENDS."  Deficiency  reports  will  be  numbered  consecutively 
(2)  The  deficiency  report  shall  contain  the 
following  information  as  applicable* 

(a)  Non-avatlability  of  the  test  material  if 
untimely  delivery  of  material  will  delay  scheduled  project 
operations.  Indicate  what  action  was  taken  by  the  prosecu¬ 
ting  command  to  eiiect  delivery  and  reasons  for  delay. 
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(b)  Kon-readiness  of  the  equipment,  as 

* 

delivered,  for  immediate  conduct  of  evaluations,  including 
a  brief  summary  ot  deficiencies  of  equipment,  action  taken 
to  correct  them,  man-hours  required  to  make  equipment  oper¬ 
able,  and  outside  assistance  required. 

(c)  Information  wherein  equipment  is  mani¬ 
festly  unsuitable  for  evaluation  due  to  major  design  defi¬ 
ciencies,  nravely  inadequate  performance,  or  other  specified 
reasons.  Include  recommended  action. 

(3)  COMOPTEVFCR  may,  in  turn,  submit  a  deficiency 
report  to  the  cognizant  SYSCOM,  activity,  or  office  concerned 
with  the  material,  with  an  information  copy  to  CNO,  the  Chief 
of  Naval  Material,  and  the  activity  reporting  to  COMOPTEVr'OR. 
COMOPTEVFOR  deficiency  reports  will  be  numbered  consecutively, 
d.  Preliminary  Report 

(1)  For  projects  which  require  an  early  report  to 
CNO  to  furnish  data  on  which  material  procurement  decisions 
are  to  be  made,  a  preliminary  report  will  be  submitted  by 
COMOPTEVFOR .  When  required  by  the  project  plan,  this  report 
will  be  prepared  and  submitted  to  COMOPTEVFOR  by  the  prose¬ 
cuting  command  by  letter  or  message  as  soon  as  preliminary 
daca  reduction  indicates  that  valid  conclusions  and  recom¬ 
mendations  can  be  made. 

(2)  A  precifa  format  for  the  preliminary  report  is 
not  specified.  Where  esults  are  of  a  very  tentative  nature 
it  may  be  essentially  narrative  inf  style.  In  instances  where 
results  are  virtually  complete,  then  the  preliminary  report 
shall  be  subdivided,  as  feasible,  into  DESCRIPTION,  RESULTS, 
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OPERATIONAL  APPLICATIONS.  CONCLUSIONS,  and  RECOMMENDATIONS. 
In  all  preliminary  reports  the  first  paragraph  will  include 
the  statement  that*  ‘‘Conclusions  and  recommendations  are 
based  on  incomplete  analysis  of  the  data  and  are  subject 
to  verification  and  possible  revision  when  data  analysis 
is  complete."  Any  deviation  from  the  preliminary  results, 
conclusions  and/or  recommendations  set  forth  in  the  preli- 
report  shall  be  noted  in  the  next  formal  report  on  the 
project. 

e.  Progress  Report 

(1)  COtoOPTEVFGR  may  direct  that  a  progress 
report  be  submitted  by  the  activity  designated  to  prosecute 
a  project  when  any  of  the  following  conditions  exists: 

(a)  The  prosecution  of  a  high  priority 
project  requires  the  reporting  of  early  and/or  continuing 
results  to  CNO  in  order  to  provide  data  for  a  decision  in 
connection  with  material  procurement, 

(b)  It  ia  necessarv  or  advisable  to  invite 
the  attention  of  CNO  to  the  status  of  a  project  or  special 
circumstances  surrounding  it,  or  to  present  limited  evalua~ 
tion  results  from  tests  to  date, 

(c)  An  extended  period  of  time  .ia  required 
to  complete  a  complex  project  which  is  not  subdivided 
readily  into  separata  parts,  tasfcs  or  phases. 

(d)  Unresolved  difficulties  in  the  analysis 
of  a  project  are  to  be  reported. 

(e)  The  prosecution  efforts  uncover  elements 


of  inconsistency,  inappropriate  coordination  or  pj-nning 
or  inadequacy  of  assistance  from  supporting  agencies  or 
systems  commands. 
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(e)  Conclus., -'ns  and  recommendations  shall 
be  included  in  the  report  provided  that  test  results  are 
considered  sufficiently  valid. 

(f)  When  the  report  involves  equipment, 
suitable  identifying  or  descrntive  photographs  and  dia¬ 
grams  may  be  included  as  -.closures. 

f .  Partial  Report 

(1)  A  partial  report,  prepared  tor  printing,  shall 
£>e  submitted  to  COMOPTEVFOR  by  the  prosecuting  command  upon 
completion  of  well  defined  part.-,  phases,  or  tasks  of  a 
project  or  if  a  final  report  on  the  entire  project  cannot 
oe  submitted  within  a  reasonable  perio”’. 

(2)  A  partial  report  is  vtpected  to  stand  as  part 
of  the  completed  record  of  the  project,  and  the  information 
contained  in  this  report  should  not  he  repeated  in  the  final 
report  other  than  to  summarize  the  major  results,  conclusions 
and  recommendations.  This  summary  shall  be  included  in  the 
"Previously  Known  Data"  section  of  the  final  report. 

(3)  The  content  and  format  to  be  used  in  preparing 
a  partial  report  are  the  same  as  for  a  final  report.  They 
are  described  in  paragraphs  508  and  509. 

(4)  Partial  reports  shall  be  numbered  in  sequence 
for  each  project. 

g.  Final  Report 

(1)  Upon  completion  of  each  Operational  Evaluation, 
Concurrent  Evaluation,  Operational  Assis  t.  Operational 
Appraisal,  and  Fleet  Operational  Investigation,  a  proDOsed 
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final  report,  prepared  for  printing,  snail  be  submitted  to 
COKOPTEVFOR  by  the  prosecuting  command* 


(2)  If  t%e  project  can  be  covered  by  a  single 
report,  that  repaid  will  ba  a  final  report  and  shall  indi¬ 
cate  that  the  project  is  concluded. 

(3)  Although  the  data  submitted  in  previous  par¬ 
tial  reports  should  not  be  duplicated  in  the  final  report, 
the  body  of  the  final  report  on  a  project,  for  which  previous 
reports  have  been  submitted,  must  be  complete  in  itselt. 

Data  summaries  and  graphs  are  required  to  provide  the  logical 
and  complete  development  of  the  final  report  and  shall  be 
included. 

h.  Supplementary  Report. 

A  supplementary  report  shall  be  submitted  under  the 
following  conditions:. 

(1)  In  those  cases  where  additional  significant 
results  become  available  after  the  submission  of  the  final 
report  on  a  project,  even  though  the  project  may  have  been 
canceled  or  terminated  by  CNO.  This  type  of  a  supplementary 
report  shall  be  prepared  in  letter  form  following  the  format 
for  a  progress  report. 

(2)  For  reporting  additional  tasks  assigned  by  the 
Chief  of  Naval  Operations  after  COMOPTEVFOR  has  submitted 

a  final  report  on  a  project,  or  upon  reactivation  of  a  pre¬ 
viously  canceled  or  terminated  project.  This  type  of 
supplementary  report  shall  be  prepared  for  printing  following 
the  outline  for  a  final  report. 
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(3)  In  those  cases  where  the  information  reported 
is  of  a  higner  classification  than  the  final  report.  This 
type  of  report  may  be  submitted  either  as  a  letter  following 
the  format  for  progress  reports  discussed  in  article  ^0*1, 
e.or,  if  of  sufficient  length,  prepared  for  printing 
following  the  outline  for  the  final  report. 

i.  Reports  on  Technical  Evaluations,  Development 
Assist  Teste .  and  Fleet  Research  Irvestications . 

Reports  will  be  submitted  to  COMOPTEVFOR  by  the 
subordinate  activity  to  which  the  project  was  reassigned 
by  COMOPTEVFOR. 

(1)  Upon  completion  of  the  project,  a  report 
summarizing  the  assistance  and  services  provided  and  con¬ 
taining  a  request  that  the  project  be  terminated  shall 

be  submitted.  Comments  and  recommendations  which  may  have 
been  previously  specified  will  ilso  be  indicated. 

(2)  Upon  assignment  of  an  assist-typ:-  project,  the 
cognizant  Headquarters  Division  Director  will  screen  the 
project  to  determine  if  it  is  of  such  importance  as  to 
warrant  submission,  to  COMOPTEVFOR,  of  a  summary  report. 

It  is  recognized  that  some  assist-type  projects  will  not 
be  of  such  importance.  If  a  summary  report  is  considered 
necessary,  the  requirement  for  its  submission  for  its  sub¬ 
mission  will  be  stated  in  the  reassignment  letter  or  message. 
This  report  will  be  submitted  no  later  than  20  working  days 
after  completion  of  the  project. 
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j»  Reports  on  Tactical  Procedures  Incident  to  OPFVAI.. 

Tactical  procedures  for  use  cf  new  or  standard  equipment, 
weapons  and  systems  developed  during  the  prosecution  of 
Operational  or  Concurrent  Evaluation  Projects  shall  be 
prepared  following  the  general  outline  format  used  in 
NWIP-1-4  or  in  the  format  of  a  change  to  the  appropriate 
warfare  publication  or  aircraft  tactical  manual.  Although 
tactical  procedures  shall  normally  be  forwarded  to  COMOPTEVFOR 
as  enclosures  to  the  reports  submitted,  the  specific  manner 
for  their  submission  shall  be  stated  in  the  project  plan. 

The  typing  of  tactical  manual  changes  to  formal  reports 
tends  to  delay  both  the  formal  report  and  the  submission  of 
changes  to  a  tactical  manual .  The  method  specified  for  the 
submission  of  tactical  procedures  should  ensure  early  intro¬ 
duction  into  the  Fleet  and  permit  early  changes  or  additions 
to  existing  manuals. 

k.  Development  cf  Tactics.  When  a  project  requires  the 
development  of  tactics  e.j  well  as  the  evaluation  of  the 
tactics,  the  presentation  of  the  tactics  developed  shall  be 
written  as  in  j. 

(1)  The  tactical  manual  or  proposed  change  shall 
normally  be  an  enclosure  to  the  basic  report;  however,  the 
manner  of  submission  will  be  specified  in  the  project  plan. 

(2)  No  abstract  of  the  presentation  is  required. 
However,  the  letter  of  transmittal  shall  explain  briefly 
the  function  of  the  manual  or  proposed  chanie. 
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Advance  Evaluation  Notes, 

AEN's  (Advance  Evaluation  Notes)  are  summaries  of  -  - 

operational,  training,  or  maintenance  information  concerning 
a  weapon  or  support  system,  component,  equipment  or  material 
under  evaluation, 

a.  The  purpose  of  the  AEN  is  to  provide  timely 
operational,  training,  or  maintenance  information  to  Fleet 
units  equipped  or  about  to  be  equipped  with  a  particular 
system  or  equipment.  Thus  the  requirement  for  AEN's  will 
exiat  in  the  following  instances: 

(1)  When  a  weapon  or  support  system,  component, 
equipment,  or  material  undergoing  evaluation  is  already  in 
the  Fleet. 

(2)  When  a  weapon  or  support  system,  component, 
equipment.-  or  material  is  nearly  iready  for  Fleet  introduction. 
Each  project  plan  promulgated  by  COMOPTEVFOR  will  contain  a 
statement  as  to  whether  AEN's  will  or  will  not  be  required. 

b.  Upon  receipt  of  an  Operational  or  Concurrent  Eval¬ 
uation  assignment,  the  cognizant  COMOPTEVFOR  Staff  Division 
Director,  or  DEPCOMOPTEVFORPAC ,  as  appropriate,  will  review 
the  assignment  details  and  determine  if  the  project  warrants 
the  promulgation  of  AEN's.  If  appropriate,  the  prosecuting 
command  shall  be  informed  of  the  frequency  of  submission  of 
proposed  AEN's  in  the  project  reassignment  letter.  AEN’s 
will  be  prepared  and  distributed  by  the  subordinate  activity 
of  OPTEVFOR  conducting  the  evaluation. 
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They  shall  be  identified  by  an  appropriate  title  and  number, 
and  written  in  an  inforaul,  narrative,  style.  AEN*s  shall 
contain  only  that  information,  obtained  as  a  result  of  valid 
tests,  which  is  considered  to  be  of  vi-luc  to  the  Fleet  for 
operational,  training,  or  maintenance  purposes.  A  discussion 
of  the  conduct  of  the  test  may  be  included  when  it  contributes 
to  a  better  understanding  of  the  AEN.  Conclusions  and 
recommendations  will  not  be  included.  Ordir.- . -ily,  A.EN’s 
will  contain  only  factual  information;  however,  tney  may 
also  contain  opinions  when  such  opinions  pert* in  to  employ¬ 
ment  methods,  procedures,  or  operating  techniques.  Opinions 
which  can  be  construed  as  pertaining  to  the  acceptability 
of  the  system  or  equipment  involved  will  not  be  included. 
Opinions  will  be  clearly  identified  as  such,  and  the  basis 
on  which  they  were  reached  will  be  indicated.  Any  informa¬ 
tion  which  would  tend  to  support  or  limit  the  validity  of 
the  opinion  will  be  included. 

c.  AEN *8  will  be  approved  by  a  review  board  from  the 
command  originatr.  the  AEN.  When  such  a  board  is  convened, 
COMOPTEVFOP  and/or  DEPCOMOPTEVFORPAC  and  cognizant  Fleet 
unit  representation  should  be  requested. 

d.  Commanding  officers  of  prosecuting1  commands  shall 
sign  all  A£N*s  or  ensure  that  they  are  properly  authenticated. 
AEN* 8  will  be  distributed  directly  by  the  preparing  activity. 

A  copy  will  be  forwarc  id  to  COMOPTEVFOR  for  information. 
Distribution  of  AEN’ a  ihall  include  Atlantic  and  Pacific 
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Fleet,  activities  which  have  a  need  for  early  information  on 
the  project.  Deployed  Pleet  commands  and  units,  the  cogni¬ 
zant  office  in  CNO,  and  appropriate  activities  of  the  Naval 
Shore  Establishment,  such  as  NAVTACDOCACT  and  the  technical 
agencies  and  laboratories  having  an  interest  in  the  project, 
shall  also  be  included. 

e.  AEN's  shall  be  reviewed  semi-annually  by  the  ori¬ 
ginating  command  in  order  to  up-date,  revise,  or  cancel  them 
as  appropriate. 

Project  Reports  submitted  by  CCMOPTEVFGR 

a.  Technical  Evaluations  and  Development  Assists. 

When  assistance  specified  in  the  project  assign¬ 
ment  has  been  provided,  a  letter  report  will  be  submitted 
to  CNO  by  COMOPTEVFOR,  with  a.  copy  to  the  agency  for  whom 
the  services  were  furnished.  The  report  will  summarize 
the  assistance  provided  and  request  termination  of  the 
project.  The  report  will  include  comments  and  recommenda¬ 
tions  when  specified  in  the  assignment  letter. 

b.  Operational  Evaluations,  Operational  Appraisal:: 
and  Fleet  Operational  Investigations. 

Upon  conclusion  of  an  Operational  Evaluation  or 
Fleet  Operational  investigation,  COMOPTEVFOR  will  submit  a 
report  of  the  finding1.:  of  such  projects  to  CNO,  with  copies 
to  appropirate  activities,  interim  reports  shall  be  sub¬ 
mitted  as  deemed  necessary  or  as  directed. 

c.  Concurrent  Evaluation. 

Upon  conclusion  of  a  Concurrent  Evaluation,  the 
developing  agency  and  COMOPTEVFOR  will  submit  reports  to  CNO. 
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(1)  The.  developing  ageney's  report  will  cover  the 


technical  aspects  of  the  evaluation,  as  requested  by  the 
Chief  of  Naval  Operations. 

(2)  COMOPTBVFOR ' s  report  will  cover  the  operational 
aspects  of  the  evaluation. 

d.  Operational  Assist. 

Upon  conclusion  of  an  Operational  Assist  COMOP-’ 
TEVFOR  will  submit  a  report  of  the  project  to  CNO  and  to 
the  developing  agency.  The  report  will  be  limited  to  the 
results  of  the  tests  conducted. 

e.  Fleet  Research  Investigations 

When  the  assistance  specified  in  the  project 
assignment  has  been  provided,  a  report  will  be  submitted  to 
CNO  by  the  Fleet  Commander-in-Chief  or  by  COMOPTEVFOR,  as 
appropriate,  with  a  copy  to  the  agency  for  whom  the  services 
were  furnished.  The  report  will  summarize  the  assistance 
provided  and  request  termination  of  the  project.  The  report 
will  include  comments  and  recommendations,  if  specified  in 
the  assignment  letter. 

f .  Letter  Extracts 

Upon  receipt  by  COMOPTEVFOR  of  reports  which  will 
be  printed,  a  letter  extract  will  bo  prepared  by  the  appro¬ 
priate  Staff  Division  for  forwarding  to  CNO.  The  letter 
extract  will  contain  the  abstract,  major  conclusions  and 
major  recommendations  from  the  basic  report  and  shall  be 
limited!  to  three  pages.  The  extract  will  be  routed  within 
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the  COMOPTEVFOR  Staff  with  the  report  from  which  extracted. 
Ten  copies  of  the  extract  report  will  be  designated  for 
the  cognizant  CNO  Project  Monitor  by  OP  number,  five  copies 
to  the  cognizant  systems  command,  and  a  copy  to  the  prose¬ 
cuting  command. 

Distribution  of  COMOPTEVFOR  Project  Reports 
Distribution  of  COMOPTEVFOR  project  reports  shall  be 
governed  by  the  current  series  of  OPNAV  Instructions  5510.1, 
005510.48  and  NAVMATINST  4000.17.  As  the  distribution  list 
varies  from  one  report  to  another,  cognizant  Project  Officer 
will  prepare  a  realistic  initial  distribution  list, 
assigning  copies  only  to  D0D  agencies  which  have  a  relevant 
interest  in  the  report.  Fifteen  copies  only  will  be  desig¬ 
nated  for  CNO  under  the  "Copy  to**  distribution,  and  the  OP 
number  of  the  CNO  Project  Monitor  as  designated  in  the 
assignment  letter  must  be  indicated.  The  CNO  Project 
Monitor  will  make  further  required  distribution  within 
QPNAV.  Other  "Copy  to"  addressees  shall  be  listed  as 
specific  individual  addressees  or  by  Standard  Navy  Distri¬ 
bution  List  numbers. 

Composition  of  EfralOfttion  Project  Reports 
The  format  to  be  employed  in  the  preparation  of  eval¬ 
uation  project  reports  is  designed  to  afford  an  orderly 
presentation  of  information,  analysis,  conclusions  and 
recommendations.  The  form  and  content  of  progress,  partial, 
final  and  supplementary  project  reports  may  vary  depending 
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upon  whether  the  report  concerns  equipment  or  tactics, 
the  format  prescribed,  however,  is  not  rigid.  It  is  recog¬ 
nized  that  certain  projects  may  be  more  effectively  reported 
if  the  format  is  varied.  Such  variations  may  be  approved 
by  the  Deputy  Chief  of  Staff  for  Operations. 

a.  Any  modifications  or  relocation  of  standard 
equipment  or  material  shall  be  described  in  detail.  Attach¬ 
ments  such  as  photographs,  diagrams  or  specifications  should 
be  used  to  reduce  the  length  of  descriptive  details  and 
ensure  clarity. 

b.  When  tests  are  conducted  at  testing  grounds  or 
areas,  such  areas  shall  be  listed.  When  test  results  are 
reduced  or  evaluated  by  mechanical  or  electronic  means, 
this  fact  should  be  mentioned  in  the  report. 

c.  When  recommending  an  equipment,  system  or  com¬ 
ponent  for  service  use,  consider,  and  include  in  the  report, 
any  qualifications  that  HERO  (hazards  of  electromagnetic 
radiation  to  ordnance)  and  other  such  tests  performed  in 
addition  to  OPTEVFOR  tests  may  have  revealed  as  a  restriction 
danger,  hazard  or  limitation. 

d.  For  all  projects,  include  run-by-run  listings  or 
graphs  of  raw  data,  and  a  matrix  presentation  of  variable 
combinations  tested,  with  sample  sizes  for  each  combination. 
On  complex  projects  involving  reduction  of  large  amounts 

of  data,  reports  must  include  sufficient  raw  data  to 
substantiate  the  results,  conrlusions  and  recommendations 
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AJVR 

APL 

AUTEC 

CINCIANTFLT 

C1NCPACFLT 

CNO 

Com  Des  Dev  Gru 
COMFAIR 

Con  Nav  Air  Iant 

COMNAVBASE 

COMTIAVORDSYSCOM 

CX»!OPTE'/H)R 

Com  Sub  Dev  Gru 

Com  Sub  Flot 

D.A. 

DCNO  (D) 

Eng  Sttt 

ETR 

fmsaeo 

jax 

KA3 

KAVKVTOOM 

NAVTACDOCACT 


Atlantic  Fleet  veapons  Range 
Applied  Physics  Laboratory 
Atlantic  Undersea  Test  &  Evaluation  Center 
Commander-in-Chief,  U.S.  Atlantic  Fleet 
Commander-in-Chief,  U.S.  Pacific  Fleet 
Chief  of  Naval  Operations 
Commander  Destroyer  Development  Group 
Commander  Fleet  *.ir 

Commander  Naval  Air  Forces,  U.S.  Atlantic  Fleet 
Commander  Naval  Base 

Commander  Naval  Ordnance  Systeris  Command 
Commander  Operational  Test  and  Evaluation  Force 
Commander  Submarine  Development  Group  • 
Commander  Submarine  Flotilla 
Developing  Agency 

Deputy  Chief  of  Naval  Operations  (Development) 
Engineering  Station 
Electron? cs  Technician 
Euetcm  Test  Range 

Fleet  Missile  Support  and  Evaluation  Group 

Jacksonville 

Naval  Air  Station 

Naval  Material  Coiaraand 

Navy  Tactical  Doctrine  Activity 
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NLON 

Nev  London 

NMC 

Naval  Missile  Center 

NOLTF 

Naval  Ordnance  Laboratory  Test  Facility 

HOTS 

Naval  Ordnance  Test  Station 

NOU 

Naval  Ordnance  Unit 

NWS 

Naval  Weapons  Station 

OEG 

Operations  Evaluation  Group 

ONR 

Office  of  Naval  Research 

OP  CON 

Operational  Control 

CPTEVFOR 

Operational  Test  and  Evaluation  Force 

PMR 

Pacific  Missile  Range 

QH 

Quartermaster 

RAV 

Restricted  Availability 

HDTfcE 

Research,  Development,  Test  and  Evaluation 

PM 

Radioman 

T AV 

Tender  Availability 

Ty  Com 

Type  Commander 

USCG 

United  States  Coast  Guard 

VACATES 

Virginia  Capes 

YN 

Yeoman 
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MARINE  CORPS  OPERATIONAL  TEST  AND  EVALUATION 


The  Marine  Corps,  wherever  possible,  utilizes  the  operational 
test  results  cf  the  other  Services.  Equipments  peculiar  to  the  Marine 
Corps  are  tested  and  evaluated  and  the  results  are  sent  to  the  Commandant 
ot  the  Marine  Corps. 

1.  ORGANIZATION:  The  Marine  Corps  docs  not  have  within  its 
R&D  oiganization  a  unit  that  is  dedicated  solely  to  Operational  Test 
fc  Evaluation  (OT&E).  The  intent /objectives  of  OT&E  are  accomplished 
by  other  'ypes  of  tests  with  the  results  utilized  in  the  subsequent  develop- 
ment/pr  jcurement  decisions.  In  lieu  of  formalized  OTiiE  tests,  the 
Marine  Corps  employs  expanded  service  tests,  troop  tests,  and  special 
operational  evaluations.  Tasking  of  Commanding  General,  Marine  Corps 
Development  and  Education  Command  (CG,  MCDEC)  for  these  tests  is 
done  by  Commandant  of  the  Marine  Corp3  (CMC)  through  the  Deputy 
Chief  of  -Staff  (RDkS).  The  nature  of  the  item  or  concept  to  be  tested 
determines  the  designation  of  the  participating  units. 

•Without  a  formal  OTf/E  command  like  the  Navy’s  Operational 
Test  &  Evaluation  Forces  (OPTEVFOR)  the- Marine  Corps  is  forced  to 
#  use  Fleet  Marine  Force  (FMF)  units  for  its  tests.  Within  the  Marine 
Corps  Development  and  Education  Command  the  Deputy  for  Development 
(Director,  Development  Center)  develops  test  plans  in  concert  with  the 
FMF  units  involved.  The  sccpe  of  testin':  and  resources  required  is 
dependent  on  the  type  of  test  and  the  nat  :  of  the  equipment/concepts 
being  tested.  CG,  MCDEC  prepares  the  requisite  test  reports  along 
with  pertinent  reuommentiations  and  submits  them  to  CMC  for  approval. 

In  those  joint  programs  where  the  Army,  Air  Force  or  Navy  docs 
the  development  and  testing,  the  Marine  Corps  participates  as  required 
and  provides  appxopriate  resources  (funds,  personnel  and  equipment). 

When  another  Service  has  statutory  responsibility  for  developmental 
action  that  also  satisfies  a  Marine  Corps  requirement,  the  attendant 
OTRrF.  is  usually  accepted  by  the  Marine  Corps  as  the  basis  for  procure¬ 
ment.  Variations  in  supply  and  maintenance,  normally  minor  in  nature, 
are  resolved  during  the  first  year  of  use.  For  major  items  of  equipment 
(aircraft,  tanks,  armored  amphibians,  weapons)  the  Marine  Corps 
depends  on  the  larger  Services  for  the  technical  development,  and  in 
most  cases  the  operational  testing,  with  varying  degrees  of  participation 
by  the  Marine  Corps. 
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2. 


LINES  OF  COMMAND:  (See  Figure  1. ) 


3.  FACILITIES  AND  ASSETS:  The  Marine  Corps  Development 
Center  located  at  Quantico,  Virginia,  and  subordinate  to  MCDEC,  is 
the  principal  RDT&E  field  activity  of  the  Marine  Corps.  A  subordinate 
test  facility  located  at  Camp  Pendleton,  California,  is  designated  the 
West  Coast  Test  Branch,  Mobility  and  Support  Division,  Development 
Center,  MCDEC.  The  Quantico  facilities  possess  limited  assets 
relative  to  OTS.-E  and  are  oriented  primarily  towards  the  administrative 
aspects  of  RD&S  (e.  g.  ,  conduct  of  studies,  development  of  requirements 
documentation,  supervision  of  Marine  Corps  sponsored  R&D  programs, 
supervision/coordination  of  joint  R&D  programs,  monitoring  of  other 
Services'  R&D  programs  and  the  development  of  tactical  doctrines). 
Ranges  and  facilities  are  also  available  for  limited  service  testing  of 
equipment. 

The  West  Coast  Test  Branch,  located  at  Camp  Pendleton, 

California,  is  oriented  and  equipped  for  the  conduct  of  service  tests 
on  equipment  peculiar  to  the  Marine  Corps  and  primarily  for  employ¬ 
ment  in  the  amphibious  environment  (ship  to  shore).  Some  testing 
of  Army  developed  vehicles  is  performed  for  the  purpose  of  determin¬ 
ing  their  suitability  for  Marine  Corps  use  in  an  amphibious  environment. 
Minimum  personnel  and  equipment  resources  are  available  at  these 
facilities.  As  previously  stated,  the  facilities  and  personnel  of  FMF 
units  are  utilized  for  many  tests. 

4.  FUNDING  :  Funding  requiremente  to  support  Marine  Corps  OT&E 
are  routinely  included  in  the  total  DOM  RDT&E  appropriations  as  part 

of  the  project  or  clement  they  are  to  support.  There  is  no  specific 
Program  Element  dedicated  to  OT&E.  Upon  approval  and  apportionment 
to  the  Marine  Corps  of  requested  program  funds,  a  similar  apportionment 
of  necessary  funds  is  provided  to  the  CG,  MCDEC  to  support  service 
and  troop  tests.  MCDEC  requirements  are  based  on  an  annual  budget 
request  from  that  facility. 

5.  SCOPE  OF  TESTING:  As  previously  explained,  in  lieu  of 
formalized  OT&E  tc'Us,  the  Marine  Corps  employs  expanded  service 
tests,  troop  tests  and  special  operational  evaluations.  The  scope  of 
those  tests  include,  but  are  not  limited  to,  the  following  specific 
determinations: 
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a.  Durability  -  Provides  data  relative  to  failure  rates, 
human  engineering  factors  and  operational  cost. 

b.  Maintainability  -  Provides  data  relative  to  the  matn- 
tenanc  i  time  required  to  support  the  vehicle  for  a  certain  period  of 
operational  time. 

c.  Fuel  and  Oil  Consumption  -  Provides  data  relative  to 
the  tactical  considerations  of  operating  range  and  battlefield  day 
capability  and  operational  costs. 

d.  Stowage  -  Provides  data  relative  to  stowage  arrange¬ 
ment  of  On  Vehicle  Equipment  (OVE)  and  the  cargo  handling  and 
securing  provisions  (where  appropriate). 

e.  Kit  Evaluation  -  Provides  data  relative  to  the  installa¬ 
tion  and  functional  aspects  of  various  kits  (where  appropriate). 

f.  Compatibility  with  Amphibious  Shipping  -  Provides  data 
reu.tive  to  compatibility  of  equipment  with  the  various  types  of 
amphibious  ships/crafts  utilized  in  amphibious  operations. 

g.  Troop  Tests  -  Provides  data  relative  to  required  design 
changes  prior  to  initiating  procurement.  Tables  of  Organization  and 
Equipment,  and  tactics  and  techniques. 

h.  Special  Operational  Evaluations  •  Provides  data  relative 
to  the  operational  potential/feasibility  of  commercially  developed  equip 
ment  prior  to  initiation  of  procurement. 

6.  TEST  REPORTS  AND  DISTRIBUTION: 

a.  Formal  reports  are  prepared  by  the  CG,  MCDF.C  on  all 
assigned  test  projects  and  submitted  to  CMC  for  review  and  approval. 

In  the  case  ol  short-termed  projects,  a  final  report  is  submitted  at 
the  completion  of  required  testing;  however,  in  the  case  of  extended 
projects,  interim  progress  reports  are  submitted  periodically  witli 
a  fin.u  report  upon  completion  of  overall  test  objectives. 
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b.  Distribution  of  reports  is  made  to  CMC,  major  FMF  commands. 
Navy  Amphibious  Fleet  Headquarters,  U.  S.  Arr.iy  Commodity  Command 
Headquarters,  U.S.  Air  Force  TAC  Headquarters,  Navy  Systems 
Commands  (where  pertinent),  all  Marine  Corps  Liaison  Officers  located 
with  other  Services  and  Defense  Documentation  Center. 


C-t>4 


APPENDIX  D 


OPERATIONAL  TESTING  AND  EVALUATION 


IN  THE  U.  S.  AIR  FORCE 


V-I 


INTRODUCTION 


This  Appendix  is  devoted  to  an  overview  of  Operational  Testing 
and  Evaluation  as  it  functions  in  the  Air  Force.  Included  are  a  brief 
account  of  the  relatively  short  history  of  OT&E  in  the  USAF,  a  few 
words  on  the  current  organisation  responsible  for  OT&E  at  Air  Staff 
level,  and  a  description  of  the  Air  Force  testing  cycle  and  the  types 
of  test  of  which  it  is  composed.  As  will  be  seen,  the  doctrine  and 
methodology  of  OT&E  in  the  Air  Force  has  resulted  from  an  evolutionary 
process,  seldom  static  but  continually  endeavoring  to  be  responsive  to 
user  needs. 


HISTORY  OF  AIR  FORCE  TESTING 


The  early  history  of  Air  Force  OT&E,  or  for  that  matt*-,  any 
type  of  evaluation  except  laboratory  testing,  centers  around  the 
organizations  and  facilities  at  Eglin  AFB.  In  1934,  the  Baker  Board 
recommended  the  creation  within  the  Army  Air  Corps  of  a  "Separate 
Branch  for  Research  and  Flight  Testing.  "  The  Chief  of  Air  Corps 
iu  1939  approved  consideration  of  an  area  at  and  near  what  is  now  the 
Eglin  AFB  complex  in  which  the  proposed  Air  Corps  proving  ground 
would  be  located.  The  geographical  features  of  the  area,  plus  the 
fact  that  a  large  part  of  it  was  already  Government-owned  land  (the 
Choctawhatchee  National  Forest)  dictated  serious  attention  to  its 
consideration.  After  considerable  development  study,  the  Air  Corps 
Proving  Ground  was  established  with  headquarters  at  Eglin  Field 
n  May  1941. 

Ttie  P.oving  Ground  contained,  in  addition  to  administrative 
functions,  a  Proof  Department  physically  stationed  at  Maxwell  Field, 
Alabama,  to  plan  test  programs  and  facilities  and  to  analyze  test 
results.  The  Proving  Ground  Detachment  at  Eglin  was  organized  to 
conduct  tests  related  to  flying.  The  Air  Corps  Board,  the  President 
of  which  was  also  Commanding  Officer  of  Eglin  Field,  was  intended  to 
be  an  operational  control  agency,  composed  cf  a  relatively  small  group 
of  highly  qualified  personnel  whose  primary  job  was  to  conduct  specialized 
studies  and  give  guidance  to  test  activities. 

Jurisdictional  differences  over  control  of  Eglin  Fie’d  and  the 
Choctawhatchee  National  Forest  dcl«  yed  real  progress  until  March  1942, 
when  reorganization  of  the  Army  Air  Forces  resulted  in  a  redesignation 
as  the  Air  Corps  Proving  Ground  Command,  By  early  1943,  however, 
the  Proving  Ground  Command  had  expanded  its  functions  to  include 
ordnance  detachments  at  Aberdeen,  Maryland;  Edgewood  Arsenal, 
Maryland;  Madison,  Indiana;  and  Hope,  Arkansas;  an  electronics 
proving  ground  at  Florosa,  Alabama;  and  an  Arctic,  Desert,  and 
Tropic  Information  Center, 

In  the  meantime,  an  AAF  School  of  Applied  Tactics  had  been 
established  to  test  the  tactical  suitability  of  equipment  which  the 
Proving  Ground  had  found  to  meet  military  requirements  (operationally 
suitable).  Interface  between  these  two  agencies  was  defined  by  a 
d.rective  issued  in  April  1943  b/  General  Arnold, 
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Between  1943  and  early  1946,  the  AAF  Proving  Ground  Command 
became  a  satellite  of  the  Army  Air  Forces  Center,  added  four  detach¬ 
ments,  activated  an  experimental  guiued  missile  group,  and  had 
replaced  the  AAF  center  at  Orlando  Army  Air  Base,  with  the  activity 
at  Eglin  reverting  to  subordinate  status.  In  July  1946,  the  AAF  Proving 
Ground  Command  was  redesignated  the  Air  Proving  Ground  Command 
with  headquarters  at  Eglin.  Following  a  series  of  internal  organizational 
changes  in  the  next  two  years,  the  APGC  was  transferred  to  the  Air 
Materiel  Command  and  for  six  months  in  1948  was  known  as  the  Air 
Materiel  Proving  Ground.  In  June  1948,  it  reverted  to  its  former  status 
as  the  Air  Proving  Ground  Command. 

The  Command's  mission  remained  essentially  the  same  throughout 
the  next  ten  years,  although  changes  in  internal  organizational  structure 
wefe  made  from  time  to  time.  Essentially,  the  Air  Proving  C  round 
Command  served  as  an  independent  testing  agency  for  the  Air  Force 
to  assess  the  Product  of  the  developer  as  to  its  "operational  suitability." 
Personnel  with  operational  experience  manned  its  testing  organization 
(3200th  Proof  Test  Group)  ar.d  tests  were  designed  and  conducted  to 
evaluate  systems  and  subsystems  in  an  environment  as  close  t<.  opera¬ 
tional  as  could  be  simulated.  Occasionally,  support  from  the  jsing 
commands  was  obtained  in  the  form  of  personnel  and  equipment  which 
were  then  included  in  realistic  simulations  of  combat  operations.  The 
dev»lopment  of  tactics  and  techniques  for  operational  employment  was 
made  a  part  of  each  test  when  and  wherever  possible.  In  short,  APGC 
served  in  a  role  as  unbiased  as  couM  be  established  between  the 
developer  and  the  user. 

The  findings  of  APGC  tests  were  not  always  welcomed  by  the 
using  commands,  since  the  latter  were  of  the  opinion  that  they  were 
in  a  better  position  to  evaluate  an  item  in  the  environment  in  which  it 
would  ultimately  be  employed.  As  a  counter  to  this,  however,  it  could 
be,  and  often  was,  maintained  that  the  Air  Proving  Ground  Command 
was  better  fitted  to  produce  quantified  da.s  winch  were  valuable  in 
decision-making  or  in  precise  evaluations  .'{  a  system's  suitability. 

In  lato  1957,  the  decision  was  made  by  He-.^quarterr,  USAF, 
to  place  APGC  under  the  cognizance  of  the  Air  Force  Research  and 
Development  Command  In  a  center  status,  combined  with  the  Air  Force 
Armament  Center  which  was  by  then  also  a  tenant  on  Eglin  AFB.  This 
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action  gave  APGC  a  new  mission,  oriented  toward  development.  It 
retains  that  function  today  under  the  Air  Force  Systems  Command. 

N'ow  designated  the  Armament  Development  and  Test  Center  (ADTC), 
it  shares  the  Eglin  facility  wi'.h  the  Tactical  Air  Warfare  Center  a.od 
the  Special  Operations  Fore.;  (both  of  the  Tactical  Air  Command). 

ADTC,  although  it  is  by  mission  development-oriented,  has 
supported  numbers  of  Category  1. 1  (OT&E)  tests  under  the  pressures 
of  the  Southeast  Asia  effort  simply  because  no  other  Air  Force  facility 
is  now  capable  of  getting  the  kinds  and  amounts  of  quantitative  data 
needed.  Some  compromises  with  realism  have  been  made  in  these 
cases  because  of  lack  of  air  and  land  space  to  support  required 
combat-type  flight  regimes  in  a  credible  threat  environment. 

In  1958  responsibility  for  planning,  funding  and  conducting  the 
Air  Force  OT&E  function  reverted  to  the  operating  commands.  It 
is  there  today,  defined  in  AF  Regulations  80-14  and  55-31.  The 
internal  organization  to  implement  it  varies  among  the  commands; 
i.  e.  ,  the  five  test  centers  of  Tactical  Air  Command,  the  Air  Defense 
Command  activity  at  Tyndall  AFB  and  the  Strategic  Air  Command 
missile  test  effort  at  Vandenberg  AFB,  as  examples.  Interface  with 
the  Air  Staff  on  OT&E  matters  is  provided  by  the  organization 
described  immediately  following. 

AIR  STAFF  ORGANIZATION  FOR  OT&E 

As  a  result  of  so'/eral  in-house  studies  on  the  conduct  and 
management  of  Air  Force  weapons  effectiveness  testing  in  the  1963-64 
period  it  became  apparent  that  a  focal  point  within  the  Air  Staff  was 
needed  to  provide  centralized  guidance  and  direction  to  an  admittedly 
fragmented  service-wide  effort.  An  ad  hoc  committee,  headed  by 
Brigadier  General  K.  C.  Dempster,  in  November  1964  recommended 
the  establishment  of  such  a  function  within  the  Air  Staff  which  would, 
in  addition  to  providing  the  needed  control,  determine  data  requirements, 
set  up  a  priorities  system,  provide  testing  resources,  insure  their 
economical  utilization,  and  assure  the  timely  processing  and  distribution 
of  test  rcsultc.  The  philosophy  of  de-centralization  of  responsibility 
for  test  design,  methodologies,  and  actual  conduct  was  to  be  retained, 
with  Air  Force  Systems  Command  responsible  for  Categories  I  and  II 
of  acquisition  testing  (R&D  oriented)  and  the  using  organization  perform¬ 
ing  and  reporting  on  operational  tests  and  evaluations  (Category  III  and 
onward).  Nuclear  weapon  tests  were  specifically  excluded. 
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The  recommendations  of  the  Dempster  committee  report  were 
implemented  in  an  Air  Force  Chief  of  Staff  directive  in  November  1964 
and  a  letter  in  December  1964  which  established  an  interim  Weapons 
Effectiveness  Testing  Task  Force  under  the  Directorate  of  Operational 
Requirements  in  the  Air  Staff.  The  resulting  Office  of  Primary  Responsi- 
bility  for  AFWET  was  reassigns4  to  the  Directorate  of  Operations, 

Deputy  Chief  of  Staff,  Plans  and  Operations  in  May  1965  as  the  Deputy 
Director  for  Air  Force  Weapons  Effectiveness  Testing.  In  August  1965, 
its  charter  was  clarified  and  redefined  in  a  Vice  Chief  of  Staff  letter 
to  the  major  Air  Force  commands.  Among  its  assigned  responsibilities 
was  the  preparation  of  an  Air  Force  regulation  governing  operational 
tests.  Responsibilities  for  strategic  ballistic  missiles,  nuclear  weapons, 
and  space  systems  were  specifically  omititd.  Original  manning 
authorization  was  for  twenty-four  manpower  spaces,  headed  by  a 
Brigadier  General. 

In  July  1965,  the  organization's  name  was  changed  to  Deputy 
Director  of  Operations  for  Operational  Test  and  Evaluation  and  its 
manning  authorization  was  increased  to'  a  total  of  sixty.  As  originally 
configured,  the  activity  was  composed  of  a  Programs  and  Resources 
Group  and  four  Divisions  -  Acquisition  Test,  Employment  Test, 
Electronics  Warfare  Test,  and  Theater  Air  Base  Vulnerability  Test. 

In  the  time  between  its  establishment  and  the  present,  the  organiza¬ 
tion  has  undergone  various  minor  internal  structural  changes,  none  of 
which  has  had  appreciable  influence  on  the  basic  charter  and  mission 
under  which  it  was  activated.  It  is  now  composed  cf  ( 1  >  ’n  Operational 
Test  Division,  charged  with  monitorship  of  Category  I  and  Ca  egory  II 
testing,  as  well  as  planning,  policy-making,  and  executive  management 
of  Air  Force  OT&E  programs  and  (2)  a  Test  Support  Division,  which 
provides  scientific  guidance  in  the  design  and  analysis  of  tests,  test 
resources,  an  Air  Force  OT&E  priority  system,  and  allied  functions. 

The  present  "Deputy  for"  operates  as  the  Air  Staff  office  of  primary 
responsibility  for  all  matters  involving  Headquarters,  USAF,  participation 
in  or  support  of  Air  Force  OT&E.  A  more  detailed  description  of  its 
mission  and  organic  structure  appears  in  an  attachment  to  thU  Appendix. 
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THE  AIR  FORCE  TESTING  CYCLE 


The  present  Air  Force  method  of  conducting  OT&E  is  to  have  it 
done  by  the  organization  which  will  ultimately  employ  the  weapons 
system,  subsystem  or  component  in  the  accomplishment  of  its  assigned 
mission.  As  provided  for  in  Air  Force  Regulations  80-14  and  55-31, 
the  principal  components  of  the  complete  testing  cycle  are: 

Acquisition  Testing 

Category  I  Tests.  There  are  development  tests  and  evaluations 
of  the  individual  components,  subsystems,  and,  in  certain  cases  a 
complete  system  under  control  of  the  Air  Force  Systems  Command. 

In  addition  to  qualification,  this  category  provides  for  redesign,  refine¬ 
ment,  and  re-evaluation  as  necessary  including  the  practicability  of 
using  current  standard  and  commercial  items.  These  tests  are  con¬ 
ducted  principally  by  the  contractor,  but  with  the  Air  Force  {AFSC,  usin^ 
commands,  and  support  commands)  active  participation,  evaluation  and 
control. 

Category  II  Tests.  There  are  development  tests  and  evaluations 
spanning  the  integration  of  subsystems  into  a  complete  system  in  as 
near  an  operational  configuration  as  practicable  under  control  of  Air 
Force  Systems  Command.  Suitable  instrumentation  is  employed  to 
determine  the  functional  capability  and  compatibility  of  subsystems. 
Category  II  is  an  Air  Force  effort  with  contractor  participation, 
under  Air  Force  (AFSC)  control  and  direction,  and  with  active  opera¬ 
ting  and  supporting  command  participation.  Actual  test  operation  and 
maintenance  is  performed  by  military  personnel  who  have  received 
formal  system  training. 

Category  III  Testa,  These  are  tests’  and, evaluations  of  operational 
systems  by  the  appropriate  operating  command.  They  include,  insofar 
as  possible,  teat  systems  which  incorporate  production  components  and 
support  items  and  which  are  operated  using  realistically  available 
personnel  skills  and  technical  data.  They  a'o  performed  under  con¬ 
ditions  as  near-operational  as  practicable,  Category  III  testing  is 
conducted  using  a  system  configuration  jointly  agreed  upon  by  Air 
iorce  Systems  Command  and  Air  Force  Logistics  Command,  It  is 
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the  first  class  of  testing  in  the  cycle  that  can  be  characterized  as  OT&E. 

It  is  done  in  accordance  with  a  specific  test  plan.  It  is  terminated  when 
pre-plar.ned  objectives  of  the  acquisition  plan  have  been  met. 

Category  III  Equivalent  Tests.  These  are  tests  and  evaluations 
cf  subsystems,  armament,  and  equipment  under  control  and  direction 
of  the  operating  commands.  They  have  the  same  objectives  as  Category 
III  tests  for  complete  systems  and  a.'e  subject  to  the  same  constraints 
and  philosophy. 

Folh/W-On  Development  Tests.  Systems,  subsystems,  armament, 
and  equipment  are  given  development  tests  and  evaluations  as  required 
after  completion  of  Category  II  tests.  These  R&D  tests,  are  used  to 
evaluate  corrections  of  deficiencies  previously  uncovered  and  improve¬ 
ments  in  systems  not  accomplished  during  the  normal  acquisition  cycle 
testing. 

Operational  Employment  Testing 

Employment  Tests.  This  is  pure  OT&E.  It  ia  conducted  by  the 
user  or  operating  command  to  improve  capability  or  to  evaluate  the 
extent  to  which  the  new  weapon  system  will  enhance  the  mission  capa¬ 
bility  of  the  operator.  It  is  designed  to  develop  tactics  and  techniques 
for  the  most  effective  weapon  usage,  dclinc  operational  problems  and 
support  new  requirements  and  modifications.  It  exposes  the  new 
system  or  subsystem  to  an  environment  as  realistically  operational 
as  practicable,  employing  only  the  support  situation,  personnel  skills, 
and  threat  environment  (if  possible)  which  the  test  item  can  be  expected' 
to  encounter.  Realism  in  the  evaluation  process  is  enhanced  wherever  pos¬ 
sible  by  the  use  of  production  items  in  statistically  reasonable  numbers. 

Operational  Evaluations.  These  arc  analyses  and  evaluations  of 
operational  data  already  available  to  minimize  the  costs  in  time  and 
money  of  actual  physical  test.  The  product  of  these  assessments  may 
be  used  as  a  datum  which  physical  test  may  ubc  as  a  point  of  departure 
or  as  actual  bases  for  conclusions  and  decision-making. 

Strategic  Ballistic  Missile  Systems  Testing 

Category  I  and  1?  Tests.  These  are  similar  in  objectives,  scope, 
and  conduct  to  the  Category  2  and  II  tests  of  orthodox  aircraft  weapon 
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systems,  subsystems  and  components.  They  are  predominantly  con¬ 
tractor  efforts,  with  increasing  military  participation  leading  to  the 
demonstration  that  system  design  goals  have  been  met  under  jon- 
operaticnal  conditions. 

Demonstration  and  Shakedown  (PASO)  Teste.  This  is  an  evaluation 
equivalent  to  Category  III  tests  of  other  systems.  It  is  conducted  by  the 
operating  command,  assisted  by  the  agency  having  Air  Force  engineering 
responsibility  in  an  operational  environment  using  operational  procedures. 
It  is  intended  to  refine  operational  and  logistic  procedures,  demonstrate 
basic  system  capabilities  and  limitations,  and  determine  system  stability 
for  its  intended  mission. 

Ballistic  Missile  Operational  Tests.  These  tests  expose  the 
system  to  an  environment  as  nearly  operational  as  possible  to  evaluate 
reliability  and  accuracy  under  realistic  conditions.  They  are  true  OT&E 
and  are  conducted  in  two  phases:  Phase  I,  a  basic  program  of  a  fixed 
(lumber  of  launches,  and  Phase  II,  an  extension  of  Phase  I,  but  consisting 
cf  a  much  siv.aller  number  of  launches,  allocated  at  an  annual  fire-out 
rate. 

Electronic  Warfare  Effectiveness  Tests 


These  are  operational  tests,  not  only  of  Electronic  Warfare  systems 
and  their  supporting  subsystems  but  of  the  tactics  and  techniques  which 
will  maximize  their  effectiveness  against  threat  EW  operations  and 
systems. 

Weapon  System  Evaluation  Program  (WSEP) 

This  is  a  continuing  weapons  firing  program  to  exercise  and 
C'  aluate  operational  aircraft  delivery  systems  and  weapons  to  determine 
reliability  and  effectiveness.  It  is  designed  to  provide  continuity  of 
weapon  sy3tem  effectiveness  data  by  obtaining  and  evaluating,  under  a 
single  program,  compatible  data  from  OT&E  sources.  The  programs 
are  conducted  by  the  using  commands  under  conditions  and  environments 
representative  of  the  projected  threat.  They  impact  on  such  areas  as 
indicated  requirements  for  changes  in  training,  tactics,  manning, 
logistics  support,  maintenance  and  modification  of  existing  hardware. 
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DEPICT  DIRECTOR  FOR  OPERATIONAL  TEST  &  EVALUATION 


I.  RESPONSIBILITIES 


Is  responsible  to  the  Director  of  Operations  for  planning,  directing 
end  evaluation  the  Air  Force  Operational  Test  and  Evaluation  (OT&E,  Pro- 
gran. 

II.  FUNCTIONS 

A.  Operational  Test  Division  (AFXOWQ) 

1.  Develops  and/or  reviews  Category  III  and  other  operational  test 
requirements,  schedules,  and  support. 

2.  Develops,  prepares  and  issues  operational  test  directives. 

3.  Reviews  and  approves  operational  test  plans. 

4.  Coordinates  with  commands  ana  monitors  operational  tests. 

5.  Establishes  and/or  approves  operational  testing  Eilestrnes  and 
program  changes. 


<  6.  Identifies  and  calces  recommendations  for  operational  testing 

<4  resources  and  priority  requirements. 


-*• 
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7.  Reviews  and  approves  operational  test  reports  and  establishes 
reports  distribution. 

8.  Through  test  and  evaluation,  develops  criteria  and  provides  inter¬ 

face  and  expertise  for  the  deployment  and  initial  employment  of  new  systems, 
equipment  and  munitions.  £ 

** 

9.  Provides  interface  and  takes  action  with  respect  to  Directorate 

positions  on  systems,  equipment,  and  munitions  during  all  phases  of  the  acqui-  r 
sition  cycle.  r 

10.  Reviews  and  keep  abreast  of  new  research  and  development  programs. 

11.  Monitors  Category  I  and  Category  II  testing  connected  with  new  /r 

equipment  and  systems  being  acquired  to  satisfy  operational  requirements.  , 

'•} 

12.  Provides  chairman  for  Trl-Servicc  Group  on  EW  Testing  Resources. 

B.  Tool  Support  Dlvlsiofi  (AFXOWG) 


3 
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1.  Pcrfoims  research  und  studies  required  to  validate  new  operational 
test  requirements. 


2, 


.  Ti  ylies  operational  problems  and  recorsaends  test  objectives. 

3.  Provides  guidance  and  direction  in  scientific  design  of  tests. 
Validates  mathematical  models  and  ca:>putar  programs  for  test  programs. 

h.  Provides  scientific  guidance  and  coordinates  operational  test 
di.rectives;  performs  quantitative  analysis  of  test  results  and  data;  and 
develops  test  methodology  and  procedures  for  Improving  test  methods  and 
concepts. 


5.  Develops  Mr  Force  test  nanngerort  ar.d  documentation  system. 

Plans  and  proviues  guidance  for  management  of  aata  management  system. 

6.  Conducts  operational  evaluations  and  analysis  cf  technological 
advancements  and  determines  operational  applications  to  current  and  fuaur' 
systems,  equipment  and  munitions. 

7.  Conducts  analytical  studies  of  Mr  Force  operational  tests  and 
problems.  Analyzes  and  evaluates  hostile  threat  for  use  in  test  design  and 
identification  of  resources  for  testing. 

8.  Maintains  data  system  on  selected  DOD  resources  and  technical 
facilities  for  use  in  Mr  Force  testing.  Requests,  monitors  and  takes  actions 
leading  to  acquisition  of  resources  and  new  technical  facilities  for  Air  Force 
OTJS. 


.. .  Develops  and  manages  a  priority  system  for  Mr  Force  OT&E  regula¬ 
tions  anu  o th.er  publications. 

10,  Servo,  as  Directorate  Office  of  Prii.-'ry  Responsibility  (OFR)  for 
Class  V  Modification  Program,  and  Required  Operat.,  onal  Capabilities  (ROCs). 
Serves  us  Mr  Staff  OPR  for  Mlocation  Priorities  uf  Unit  Resources. 

11,  Serves  as  Mr  Staff  OPR  on  matters  concerning  the  Deseret  Test 
Center.  Maintains  liaison  with  Defense  Atomic  Support  Agency  on  iluclear 
Weapons  Testing  matters  and  with  the  other  Services  and  POD  on  weapons  testing. 
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DEPUTY  DIRECTOR 

"  tor 

CTERATIOtUw  TEST  &  EVALUATION 


OPERATIONAL  TEST  DIVISION 


Develops  test  requirements,  objec¬ 
tives  and  support  requirements. 
Issues  Test  Directives.  Reviews 
and  approves  Test  Plans  and  reports 
Monitors  testing  and  R&D  prograss. 
Establishes  milestones  and  test 
program  changes.  Provides  inter¬ 
face  with  Research  and  Devclop-.ent. 
Provides  criteria  for  new  system 
deployment.  Establishes  test 
reports  distribution.  Monitors 
Category  I  and  Category  II  Test 
Programs. 


I - 

BRANCHES 

Aerospace  Systems  and  Equipment 
Weapons  and  Missiles 
Electronic  Systems 


TEST  SUPPORT  DIVIEION 


Operational  test  requirements 
and  test  objectives  validation. 
Design  of  tests.  Mathematical 
modeling  and  cccputer  programs 
for  OT&S.  Coordinates  test  dir¬ 
ectives:  performs  qualitative 
analysis  and  develops  test  metho¬ 
dology  and  procedures.  Conducts 
operational  evaluations.  Eval¬ 
uates  hostile  threat  for  test 
design  ana  resources.  Office  of 
Primary  Responsibility  (OPR)  for 
test  data  management.  Requests 
resources  and  technical  facili¬ 
ties  for  0T&2.  Coordinates  test 
directives;  performs  qualitative 
analysis  and  develops  test  metho¬ 
dology  and  procedures.  Conducts 
operational  evaluatior.3.  Evalu¬ 
ates  hostile  threat  for  test 
design  and  resources.  OPR  for 
test  data  management.  Requests 
resources  and  technical  facili¬ 
ties  for  OT&E.  AFXOP  OPH  fer 
Class  V  Modifications,  Priorities, 
Required  Operational  Capabilities, 
Southeast  Asia  Oj'erational  Re¬ 
quirements,  Deseret  Test  Center 
and  nuclear  tests, 

BRANCHES  ’*’* 

Analysis  and  Evaluation 
Resources 

Claes  V  Modifications 
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INTRODUCTION 


In  an  effort  to  gain  more  i.isight  into  OT&E.  the  staff  selected 
approximately  sixt,  military  and  civilian  personnel  to  interview  on 
the  subject.  These  individuals  were  chosen  or.  the  basis  of  their 
known  extensive  backgrounds  in  testing  or  because  of  the  present 
positions  they  were  filling  in  a  test  facility  or  staff.  The  level  of 
these  individuals  ranged  through  pilots  flying  OT&E  missions  in 
evaluation  squadrons,  heads  of  test  facilities  and  ranges,  high-level 
civilians  in  civil  service  testing  jobs,  members  of  civilian  industry 
staffs,  and  deputies  of  Service  RDT&E  programs.  A  complete  list 
of  personnel  is  shown  in  Inclosure  1. 

Some  of  these  interviews  took  the  form  of  a  briefing  on  their 
facility  and/or  the  way  they  conducted  OT&E,  followed  by  a  general 
discussion. period.  •  Others  were  strictly  a  round-table  type  discussion. 

In  order  to  insure  coverage  of  some  of  the  basic  issues,  a 
standard  list  of  questions  was  prepared  in  advance  as  a  starting 
point  for  the  discussion.  It  was  stressed  that  this  was  an  informal 
discussion,  and  no  answers  would  be  identified  with  any  individual. 

The  answers  that  were  solicited  were  not  the  official  Service  positions 
but  the  true  feelings  of  the  individual,  based  upon  his  extensive 
experience  in  the  testing  field. 

1.  Do  you  believe  that  there  is  a  need  to  change  Defense  Department 
policy  and  or,anization  to  insure  the  conduct  of  more  productive 
operational  tet  :s? 

2.  Do  yo  i  think  there  is  a  rec  lirement  for  an  OT&E  organization 
at  higher  than  Serv  <?•  level?  "  If  so,  what  should  be  the  scope  of 
its  responsibilities'. 

Within  the  Services,  tlu  !:•  rge  majority  of  individuals  interviewed 
did  not  sec  any  need  for  major  cua-  <e  i  i  Defense  Department  policy 
ana  organization  relative  to  OT&E.  The  responses  indicated  general 
satisfaction  with  the  function  within  on.  h  5-  jrviee,  and  most  importantly, 
with  the  capability  of  the  Service  to  satisfactorily  come  to  grips  with 
its  own  OT&E  problems.  The  prevalent  opinion  is  that  the  Service 
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itself  is  much  better  able  to  cope  with  operational  matters;  to  elevate 
major  OT&E  responsibilities  to  OSD  would  only  further  remove 
decision  making  from  reality  and  result  in  significant  delays  in 
decisions. 

Several  persons,  in  particular  some  external  to  the  Services, 
advocated  for  significant  change  tc  OSD  policy  and  organizations. 

The  arguments  here  varied  -  some  fell  a  large  activity  (Deputy 
Assistant  Secretary  of  Defense  for  OT<*E)  was  necessary  while  others 
specified  smaller  but  potent  activities  within  the  current  DDR&E  or 
Systems  Analysis  struct"res. 

A  key  point  raised  was  the  nature  of  responsibilities  of  any  OSD 
activity.  Most  interviewees  saw  little  objection  to  some  OT&E  function 
at  the  OSD  level.  The  Services  ter.>!  to  look  on  such  an  activity  as 
one  that  would  monitor  Service  OT&E,  be  helpful  in  obtaining  funds 
and  facilities,  and  perhaps  occasionally  refereeing  (or  suggesting) 
OT&E  activities  that  involve  more  than  one  Service.  However,  any 
OSD  function  to  direct  and  control  OT&E  was  almost  universally 
rejected  as  inherently  unresponsive  and  a  potential  negative  force 
on  OT&E  effectiveness. 

The  most  reasonable  and  productive  function  for  an  OSD  OT&E 
activity  was  considered  to  be  the  establishment  of  overall  general 
policy  and  perhaps  framing  the  pertinent  questions  about  capabilities 
that  would  assist  Services  in  developing  their  OT&E  to  also  respond 
to  valid  OSD  tiiformation  requirements. 

As  concerns  OT&E  policy  and  organization  within  each  Service, 
interviev'ecs  generally  felt  that  his  own  Service’s  way  of  accomplishing 
OT&E  was  r>-csonable  and  responsive.  Service  OT&E  organizations 
vary,  Ti  .  Navy  places  OT&E  responsibilities  in  an  independent 
organisation  (neither  developer  or  user)  reporting  directly  to  the  Chief 
of  Naval  Operations,  The  Army  has  an  independent  materiel-oriented 
OT&E  activity  under  the  developer  organizationally,  but  reporting 
directly  to  the  Chief  of  Staff.  Other  OT&E  in  the  Army,  test  and 
evaluation  of  operational  and  organizational  concepts,  is  done  by  an 
agency  which  represents  the  user  and  also  reports  directly  to  the 
Chief  of  Staff.  The  Air  Force  has  OT&E  a  user  responsibility.  As 
a  result  of  these  differences,  the  question  of  independence  and 
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reporting  level  for  Service  OT&E  was  much  discussed.  The  general 
opinion  was  the  separation  of  OT&E  from  the  developer  was  necessary. 
Opinion  on  degree  of  user  responsibility  and  participation  varied; 
however,  the  need  for  trained,  knowledgeable  and  motivated  working 
level  OT&E  personnel  was  considered  more  important  than  organization. 
There  was  some  support  for  an  independent  OT&E  activity  within  the 
Air  Force  and  reporting  to  the  Chief  of  Staff.  However,  Air  Force 
experience  with  the  Air  Proving  Ground  Command  was  fresh  enough 
so  that  the  creation  of  an. identical  act'vity  -  large  and  cumbersome  - 
was  not  considered  appropriate. 

3.  How  should  OT&E  be  funded? 

Provide  a  level  of  effort  f  inding  to  take  care  of  housekeeping  and 
indirect  labor.  Program  managers  should  budget  and  fund  for  direct 
labor  costs  of  their  testing  programs  *pd  for  any  special  instrumenta¬ 
tion  required  to  carry  r.  it  the  tests.  Procurement  funds  should  be 
used  where  an  item  is  consumed  by  the  test.  Example  -  missiles. 

It  is  felt  that  funding  should  be  consistent  between  the  Services 
and  ranges.  Customers  are  forced  to  use  range  facilities  that  are 
free  to  them  versus  ranges  that  are  industrially  funded  even  though 
the  overall  cost  to  the  Government  might  be  more.  Industrial  funding 
tends  to  make  facilities  cut  overhead  costs  to  a  minimum;  however, 
on  the  other  hand,  it  tends  to  make  the  program  manager  cut  his  test 
program  below  the  optimum  level  because  of  the  cost. 

The  general  consensu*  was  that  OT&E  was  receiving  adequate 
funds  now  for  the  type  of  OT&E  being  conducted  considering  budget 
constraints.  However,  if  OT&E  should  be  undertaken  on  the  scale 
envisioned  by  the  PSAC  report,  a  considerable  increase  in  the  T&E 
budget  would  be  required. 

4,  Should  production  other  than  prototype  be  contracted  before 

completion  of  OT&E7 

This  question  is  Influenced  by  many  other  factors,  A  weapon 
system  like  a  fightir  aircraft  will  bo  used  here  as  an  example.  The 
basic  design  of  the  F-4  aircraft  was  submitted  in  1953,  and  the  first 
contract  was  signed  in  1954,  The  aircraft  wan  tirsl  introduced  to  an 
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operational  squadron  in  1960.  The  present  replacement  for  the  F-4 
is  the  F-14,  which  should  be  finishing  operational  testing  about  1975. 

If  the  decision  is  made  to  go  into  limited  production  after  first  flight 
in  1971,  the  aircraft  would  get  to  the  fleet  in  limited  numbers  in 
1975,  or  twenty-two  years  after  the  F-4  was  conceived. 

If  the  decision  i»  made  to  test  before  buy,  and  after  testing  in 
1975  the  F-14  does  not  come  up  to  expectations,  it  would  take  an 
additional  seven  years  minimum  to  get  another  system  up  to  this 
point.  The  F-4  would  then  be  twenty-nine  years  old  and  would  have  been 
unable  to  meet  the  Russian  threat  for  many  years.  I •)  other  words, 
when  you  do  not  have  a  new  prototype  coming  along  every  two  -  three 
years,  you  are  more  or  less  locked  in  tc  making  the  new  system 
work  because  it  is  better  than  what  you  have.  If  you  have  several 
weapon  systems  in  R&D  spaced  properly,  yon  can  afford  to  go  slower, 
do  more  thorough  testing,  and  undoubtedly  get  a  better  product. 

The  general  consensus  of  those  interviewed  was  that  under  the 
present  system,  it  is  too  costly  on  major  systems  not  to  contract  for 
some  production  models  prior  to  completion  of  testing. 


5. 


For  a  major  weapon  system,  how  many  test  vehicles  are  required 
for  a  thorough  test  program? 


For  aircraft  and  tanks,  six-to-eight  would  be  the  minimum  number 
for  Operational  Evaluation  only.  For  the  complete  test  program  thi  ough 
Operational  Evaluation,  the  numbers  should  be  on  the  order  of  sixteen 
to  twenty.  For  other  items  such  as  a  major  iterr  of  complex  electronics 
equipment,  the  number  can  be  a»  low  as  two.  It  was  evident  that  the 
particular  weapon  system  being  tested  determined  the  number  needed. 
Most  people  felt  that  test  items  for  Operational  Evaluation  should  be 
manufactured  with  productior.-line  tooling  since  production-line 
techniques  will  result  in  a  different  product  than  R&D  techniques. 

6.  V/hat  arc  your  thoughts  on  test  ranges  and  their  control  - 

national  ranges,  Service  ranges,  contractor  ranges,  methods 
of  funding,  methods  of  scheduling? 

Responses  to  this  question  varied  somewhat  according  to  the 
Service  affiliation  of  the  interviewee.  In  general,  U,  S,  Arm/ rcrsoti'-.cl 
considered  their  facilities  adequate  for  OTfrE  (Service  Testing  in  the 
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Army)  as  did  U.  S.  Navy  personnel.  U.  S.  Air  Force  responses 
indicated  general  satisfaction  with  R&D  test  facilities/capahilities 
but  also  cited  OT&E  ranges  as  limited  in  proper  scope  and  instrumenta¬ 
tion.  In  this  regard,  the  Air  Force-proposed  concept  for  an  extensive 
capability  to  conduct  integrated  offensive-defensive  OT&E  with  sizable 
numbers  of  airborne  systems  in  conjunction  with  realistic  simulated 
threat  situations  (the  HAVE  EDGE  study)  was  the  subject  of  some 
comment.  Although  cuch  a  capability  was  considered  ideal  and 
certainly  of  far-reaching  potential  for  needed  mission-oriented  OT&E, 
not  all  agreed  that  the  concept  was  feasible  within  the  present  state-of- 
the-art  of  instrumentation  or  ability  to  construct  an  inclusive  and 
realistically  f  1  — cible  threat  environment. 

A  serious  problem  of  concern  to  most  interviewees,  regardless 
of  Service,  is  the  slow  but  continuing  encroachment  on  range  space 
and  facilities  by  civilian  communities,  -ivi!  air  desires,  natural 
resource  exploitation  interests,  and  other  interest  groups  within  and 
outside  of  the  Government.  Many  ranges  have  lost  some  flexibility 
to  do  comprehensive  OT&E  due  tc  flight  restrictions. 

In  general,  scheduling  of  Service  ranges  is  not  considered  a 
major  problem.  Each  Service  feels  its  ability  to  control  its  ranges 
is  essential  for  responsiveness.  In  addition,  utilization  of  ether 
Service  ranges  when  necessary  is  not  considered  a  major  problem. 
Scheduling  and  reimbursement  procedures' are  adequate  and  reasonably 
responsive. 

The  need  for  national  range  resources  is  generally  appreciated; 
however,  these  should  be  limited  to  those  unique  and  costly  resources 
of  common  use  to  all  Service  needs. 

Funding  of  ranges  was  che  subject  of  extensive  comment.  Funding 
methods  and  procedures  vary  between  and  within  the  Services.  Most 
felt  that  the  Services  should  have  the  same  type  funding  procedures. 

The  majority  opinion  was  for  some  level  of  effort  Service  funding  as 
the  best  means  of  preventing  loss  of  capability.  However,  there  was 
some  support  for  industrial  funding  as  the  best  way  to  force  planning 
and  eliminate  inefficiencies. 


Separate  funding  for  OT&E  {as  a  program  element)  was  given 
some  support.  Identification  of  OT&E  fund  requirements  by  project 
was  considered  beneficial  -  particularly  when  the  project  required 
new  facilities/capabilities. 

7.  What  kinds  of  actual  combat  experience  are  amenable  to 

evaluations  as  contrasted  with  formal  OT&E? 

The  majority  of  the  respondents  to  this  question  were  of  the 
opinion  that  actual  combat  experience  has  limited  value  as  a  source 
of  operational  data.  The  main  reasons  giver,  for  this  consensus  was 
that  the  lack  of  control  over  the  situation  in  combat  resulted  in  data 
of  unknown  validity.  Some  data  such  as  that  pertaining  to  maintenance, 
logistics,  sorties  flown  or  ordnance  delivered  can  be  accurately 
recorded,  but  data  concerning  effectiveness  cannot  be  acquired  with 
any  degree  of  assurance  regarding  its  validity. 

8.  Define  Operational  Testing  and  Evaluation. 

There  was  no  general  agreement  among  the  respondents  as  to  a 
precise  definition  of  Operational  Testing  and  Evaluation  The  most 
frequently  mentioned  elements  of  OT&E  were  operational  environment 
(actual  and  simulated),  typical  user  personnel  and  operationally  con¬ 
figured  systems  (prototype  or  early  production  models).  Purposes 
mentioned  for  OT&E  ranged  from  determining  the  suability  of  newly 
developed  systems  to  developing  new  uses  for  old  systems.  The  follow¬ 
ing  definition  is  a  distillation  of  the  majority  of  the  rtsponees  to  this 
question:  a  test  conducted  under  actual  or  simulated  ?peraticnal  con¬ 
ditions  to  determine  the  suitability  of  an  item  to  accomplish  its  intended 
function;  the  host  way  to  employ  an  item;  or  modifications  nee d ed  t o 
make  a  system  more  usef u  1 . 

Some  respondents  indicated  that  in  their  opinion  operational 
evaluation  precedes  the  development  of  requirements  and  that  subsequent 
operational  testing  constitutes  only  one  of  many  inputs  to  continued 
operational  evaluation.  Several  responses  included  comments  to  the 
effect  that  OT&E  continues  throughout  the  life  of  a  system. 
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9.  Can  OT&E  be  used  in  the  decision-making  process?  If  so,  how? 

The  majority  of  the  respondents  were  of  the  opinion  that  the 
results  of  OT&E  must  be  considered  in  production  decisions.  A 
significant  minority  maintained  that  OT&E  cannot  be  realistically 
accomplished  until  production  models  are  available  and  that  OT&E 
cannot  therefore  provide  input  to  a  decision  to  produce.  In  recognition 
of  this  minority  position,  several  respondents  indicated  a  belief  that 
production  should  be  limited  until  OT&E  is  complete.  Others  felt 
that  OT&E  of  prototypes  or  R&D  models  would  suffice  as  a  basis  for 
decisions  to  produce  a  system  in  quantity. 

10.  How  does  the  quality  of  OT&E  now  compare  to  similar  efforts 

of  a  decade  or  more  ago? 

How  does  it  compare  with  what  is  needed? 

There  appeared  to  be  substantial  agreement  among  those 
responding  to  this  question  that  OT&E  is  better  now  than  in  the  past; 
however,  there  was  a  recognition  that  there  is  still  a  great  deai  of 
room  for  further  improvement.  Areas  requiring  improvement  were 
the  use  of  more  precise  analysis  of  results,  more  use  of  compute* 
simulations  as  an  assist  to  analysis,  and  the  development  of  a  higher 
degree  of  testing  expertise  within  the  Services. 
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INDIVIDUALS  INTERVIEWED  BY  THE  OT&E  TASK  GROUP 


Joint  Chiefs  of  Staff 


BG  Russell  A.  Berg,  USAF  -  Requirements  and  Developments 
Division,  J-5  (Plans  and  Policy) 

BG  Maurice  W.  Kendall,  USAF  -  Deputy  Director  for  Co nsnands/ 
Areas,  J-3  (Operations' 

BG  John  J,  Kenney,  Jr. ,  USA  -  Western  Hemisphere  Division, 

J-5  (Plans  and  Policy) 

BG  R.  P.  Lukeman,  USAF  -  Strategic  Division,  J-5  (Plans 
ar.d  Policy' 

Army  or  Army  Affiliated 

LTC  Curtis  Amende  -  Deputy  Chief  of  Test  &  Evaluation 
Division,  Headquarters,  Army  Materiel  Command, 
Washington,  D.  C. 

LTG  Austin  W.  Betts  -  Chief  of  Research  &  Development, 
Department  of  the  Army, Washington,  D.  C. 

Mr.  ■’enjamin  ?  Goodwin  -  Special  Assistant  to  the  Commanding 
Gi..ct.al,  Test  i  ^valuation  Command,  Aberdeen  Proving 
Ground,  Maryland 

Mr.  Jack  Harris  -  Plans  and  Programs  Division.  Combat 
Development  Command,  Ft,  Belvoir,  Virginia 

LTC  John  P.  Hauraersen  -  Chief,  Reliability  &  Maintainability 
Branch,  Management  A  Evaluation  Division,  Office,  Chief 
of  Research  &  Development,  Department  of  the  Army, 
Washington,  D.  C. 

COL  E.  0,  Kitchens  *  Commanding  General,  Combat  Arms  Croup, 
Combat  Development  Command,  Ft.  Leavenworth,  Kansas 

COL  C.  J.  Molloy  Jr.,  Director,  Plans  A  Operations,  Test  & 
Evaluation  Command,  Aberdeen  Proving  Ground,  Aberdeen, 
Marvland 

COL  N.  L.  Robinson  -  Commanding  Officer,  Yuma  Proving  Grounds, 
Yuma,  Arizona 

MG  E.  L.  Rowny  -  Deputy  Chief  of  Research  &  Development, 
Department  of  the  Army,  Waslil  igton,  D.  C. 
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B3  Ceorge  Samraet,  Jr.  -  Director,  Plans  6  Programs  Office, 

Chief  of  Research  &  Development,  Department  of  the  Army, 
Washington,  D.  C. 

LTC  H.  Thaxton  -  Director,  Test  &  Evaluation,  Yuma  Proving 
Grounds,  Yuna,  Arizona 

Mr.  Floyd  Watts  -  Technical  Advisor,  Yuoa  Proving  Grounds, 
Yuma,  Arizona 

Mr.  Arthur  Woods  -  Office  of  Assistant  Vice  Chief  of  Staff, 
Department  of  the  Army,  Washington,  D.  C. 

Navy  and  Marine  Corps  or  Xavv  Affiliated 

CAPT  James  L.  Anderson  -  Director,  Surface  Warfare  Division, 
Operational  Test  &  Evaluation  Force,  t'.  S.  Hava)  Base, 
Norfolk,  Virginia 

Mr.  C.  J.  DiPol  -  Associate  Department  Head  (Operations), 
Naval  Weapons  Center,  China  Lake,  California 

CArT  James  Foster  -  Commanding  Officer,  VX-4-N/.S,  Pt.  Mugu, 
California 

CAPT  Charles  Fritz  -  Commanding  Officer,  VX-5-NAS,  China  Lake, 
California 

Dr.  1.  E.  Hlphberg  -  Head,  Systems  Development  Department, 
Naval  Weapons  Center,  China  Lake,  California 

CAPT  Leuls  A.  Hopkins  -  Commanding  Officer,  Naval  Missile 
Center,  Pt.  Mugu,  California 

CAPT  Richard  W.  Huxford  -  Dilector,  Air  Warfare  Division, 

Operational  Test  and  Evaluation  Force,  U.  S.  Naval  Aase, 
Norfolk,  Virginia 

CAPT  R.  T.  Jenlec  -  Range  Operations  Officer,  Pacific  Missile 
Range,  Pt.  Mugu,  California 

CDR  Rudolph  L.  Krause  -  Head,  Fighter  Weapon  Section,  Air 
Warfare  Division,  Operational  Test  &  Evaluation  Force, 

U.  S,  Naval  Base,  Norfolk,  Virginia 

COL  E.  S,  Maloney  -  Assistant  Deputy  Chief  of  Staff  (RD&S), 
Headquarters,  II.  S.  Mnrinc  Corps,  Washington,  D.  C. 

CAPT  J.  K.  HcConcghy,  Jr.  *  Executive  Officer,  Naval  Weapons 
Center,  China  Lake,  California 
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CAPT  J.  D.  Mooney  -  Director,  Pacific  Missile  Range 
Directorate,  Pt.  Mugu,  California 

RADM  Howard  S.  Moore  -  Canaander,  Pacific  Missile  Range, 

Pt.  Mugu,  California 

CDR  E.  C.  Parker  -  Projects  Director,  Air  Test  &  Evaluation 
Squadron  rive,  Naval  Air  Station,  China  Lake,  California 

C CL  K.  P.  Rice  -  USMC  Liaison  Officer,  IJav&l  Weapons  Center, 

China  Lake,  California 

YADM  Edward  A.  Ruckner  -  Deputy  Chief  of  Naval  Operations 
(Development),  Washington,  D.C. 

Mr.  Carl  Schaniel  -  Naval  Ordnance  Test  Station,  China  Lake, 
California 

RADM  Levering  Smith  -  Director,  Strategic  System  Projects, 
Headquarters,  Naval  Material  Coamanct,  Washington,  D.C. 

CAPT  N.  J.  Smith,  III  -  Deputy  Test  &  Evaluation  Coordinator, 
Naval  Air-  Station,  Patuxent  River,  Maryland 

CAPT  George  Watkins  -  Deputy  Director,  Naval  Air  Test  Center, 
Patuxent  River,  Maryland 

Mr.  H.  G.  Wilson  -  Deputy  Technical  Director,  Naval  Weapons 
Center,  China  Lake,  California 

Air  Force  or  Air  Force  Affiliated 

Brig.  Gen.  C.  H.  Bolender  -  Acting  Director,  Development, 

DCS/R&D,  Headquarters,  USA!',  Washington,  D.C. 

Col. Jerald  Cooke  -  International  Security  Agency,  OSD, 

Washington,  D.C. 

Mr.  Thomas  Dolehite  -  Chief  Scientist,  Armament  Development 
Test  Center,  Eglin  Air  Force  Bose,  Florida 

Erig.  Gen,  M.  W.  Elliot  -  Assistant  Director  (Ranges  &  Space 
Group  Support)  DDR&E,  OSD,  Washington,  D.C. 

Brig.  Gen.,  Frank  K.  Everest,  Jr.  -  Assistant  Director,  Operational 
Test  &  Evaluation,  DDR&E,  OSD,  Washington,  D.C. 

Lt.  Gen.  Harry  E.  Goldsworthy  -  Deputy  Chief  of  Staff  for 
Systems  and  Logistics,  Headquarters,  USAF,  Washington, 

D.C. 
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Mr.  Charles  Haro  -  Chief  of  Operational  Analysis  Section, 
Tactical  Air  Warfare  Center,  Sglir  Air  Force  Base, 
Florida 

Col.  Edward  F.  Johnston  -  Deputy  Chief  of  Staff  for  Plans  & 
Operations,  Arsed  Forces  Staff  College,  Norfolk, 
Virginia 

Col.  Kcrbeit  I.eonhardt  -  Deputy  for  Operational  Test  & 

Evaluation,  DCC/Plans  Operations,  Headquarters ,  USAF, 
Washington,  D.C. 

Col.  George  Lutz  -  Chief,  Operations  Test  Division,  Deputy 
Director  for  Operational  Test  '<  Evaluation,  DCS/plans  & 
Operations.  Headquarter;.,  USAF,  Washington,  D.C. 

Lt.  Col.  Edgar  A.  Northrup,  Jr.  -  Strategic  Division, 
Directorate  of  Operations,  Headquarters,  USAF, 
Washington,  D.C. 


Other.. 


Mr.  Charles  H.  Bernstein  -  deputy  Assistant  Secretary  ' 
(Strategic  Programs),  OSD/SA,  Washington,  D.C. 
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Mr.  C.  J.  B~ady  -  Karager,  CM  proving  Grounds,  Milford, 
Michigan 

Mr.  K.  E.  BvooV.er  -  Manager,  Engineering  staff  Operations, 
GM  Engineering  Staff,  GM  Technical  Center,  Warren, 
Michigan 

Mr.  A1  Eaton  -  Missile  Systems  Division  Supervisor, 

Applied  physics  Laboratory,  Johns  Hopkins  University, 
Silver  Spring,  Maryland 

Dr.  Robert  Fox  -  Director,  Science  &  Technology  Divis.on, 
Institute  for  Defense  Analyses,  Arlington,  Virginia. 

Mr.  Patrick  Gross  -  Special  Assistant  to  Director,  Land 
Forces  Programs  OSD/SA,  Washington,  D.C. 

Mr.  L.  A.  Kintigh  -  Vice  President  of  General  Motors 
Engineering  Staff,  Warren,  Michigan 

Col, Clifford  Moore,  Jr,,  USAF  -  Weapons  Systems  Evaluation 
Group,  DDRC<E,  Washington,  D.C. 

LTO  Arthur  W.  Oberteck,  USA  -  Director,  Weapons  Systems 
Evaluation  Branch,  DDF&E,  Washington,  D.C. 


; 

l 

V 

> 


J 


\  ' 


i 

i 

i 


1 

& 


n 


E 


E- 1 1 


Inclosure  1 


<A  -  S'-, 


Dr.  William  Pettijohn  -  President,  Vertex  Corporation, 
Kensington,  .Maryland 

Mr.  Pierre  Sprov  -  Assistant  fer  Special  Projects  k  Principal 
Deputy  Assistant  Secretary  OSD/SA,  Washington,  D.  C. 

Mr.  Richard  Stubbing  -  Budget  Examiner.  National  Security 
Programs  Division,  Bureau  of  the  Budget,  Washington. 
O.C. 


E-1Z 


Inclosure  1 


it;  '• 


I 

I 

I 

APPENDIX  F 

T 

i 

*  OT&E  CASE  HISTORIES 


i 

i 


Selected  Operational  Tests 

and 


System  Development  Programs 


CONTENTS 


Test  History  of  the  M551  (Sheridan/Shillelagh) 

Weapons  Systen  ,  a  it  Relates  to  Major  Decisions . F-2 

Test  History  of  the  Pershing  Weapons  System 

As  it  Relates  to  Major  Decisions . F-5 

Test  History  of  the  OH-6A  (Cayuse) 


Helicopter  as  it  Relates  tc  Major  Decisions .  F-8 

Troop  Test  of  Land  Navigation  Systems  •  .  .  . .  F-10 

Test  History  of  the  A -6  A  (Grumman  All  Weather 

Attack  Aircraft)  as  it  Relates  to  Major  Decisions  .  .  .  >  .  F-12 

Development  History  of  Naval  Tactical  Data  System  (NTDS)(U) 
CONFIDENTIAL .  * 

Tasting  History  of  the  C-141  Aircraft 

(Categories  I,  II,  and  III) .  F-15 

Testing  History  of  F-5  Aircraft  . . .  .  F-19 


3 


* 


| 

a 

| 


■i 


Test  History  of  COMBAT  FLIP  -  Category  III 


OT&E  of  the  AIM-9E  Missile  (U)  SECRET .  * 

Development  History  of  the  M-16  Rifle  . .  F-22 


*Not  included  here  because  of  security  classification.  They  were 
published  separately  and  made  available  to  the  Blue  Ribbon  Defense 
Panel  and  its  staff. 


U 

rt 


•> 


h 

>* 

& 


"5 


1 


INTRODUCTION 


Among  the  specific  scudy  tasks  which  the  OT&E  Task  Group 
undertook  was  documentation  of  selected  represented  OT&E  efforts. 

The  results  of  this  undertaking  are  presenter,  in  this  Appendix  in  the 
form  of  case  histories.  These  case  histories  include  both  specific 
operational  tes  and  also  development  programs  with  OT&E  activities 
ant!  their  inflt  ence  (oi  lack  thereof)  noted  and  discussed.  It  is  believed 
that  studying  these  case  histories  will  yield  insight  into  the  characteristics 
of  bo_h  successful  and  unsuccessful  OT&E  and  will  also  point  the  way  to 
actions  which  are  required  to  make  OT&E  play  a  more  important  role 
in  placing  effective  weapon  systems  and  materiel  in  the  hands  of  the 
operational  forces. 

Detailed  comments  on  specific  testing  are  found  in  the  individual 
case  histories.  T.t  should  be  pointed  out,  however,  that  there  is  a 
differ,  .ice  between  "good”  operational  testing  and  "effective"  operational 
testing.  For  OT&E  to  be  effective,  ifc  results  must  be  communicated  to 
the  proper  level  and  acted  upon.  'Thor  is  considerable  evidence  that 
such  communication  and  subsequent  action  represent  weak  links  in  the 
overall  system.  For  tn:s  reason,  it  is  evident  that  much  "good"  OT&E 
has  not  been  "effective"  OT&E. 
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TEST  HISTORY  OF  THE  f/^51  (SHER IDAN/SHILLELAGH) 
WEAPONS  SYSTEM  AS  IT  RELATES  TO  MAJOR  DECISIONS 


background 

The  \!551  weapons  system  has  been  the  subject  of  intensive 
investigation  by  Congress  and  the  General  Accounting  Office.  One 
result  of  this  investigation  has  been  an  allegation  that  the  system 
was  not  adequately  tested  prior  to  its  full  scale  procurement. 

Discussion 


The.  requirement  for  an  Armored  Reconnaissance  Airborne  Assault 
Vehicle  weapons  system  was  approved  in  1959.  This  requirement 
idertified  the  need  for  an  armored  vehicle  to  function  as  the  main 
reco  raissance  weapon  for  armor,  infantry  ami  airborne  operations, 
and  as  Ihe  main  assault  weapon  for  airborne  operations  and  combined 
arms  teams  not  employing  the  main  battle  tank.  The  vehicle  was  to 
possess  a  high  degree  of  cross -country  mobility,  have  an  inherent 
swimming  capability,  and  be  air  transportable  and  air  droppable.  The 
vehicle  was  to  replace  the  M56  Self-  Propelled  Anti-Tank  (SPAT)  Weapon 
and  the  M-41  tank.  The  VI 5 5 1  weapons  system  was  developed  to  satisfy 
this  requirement. 

The  design  which  was  adopted  for  development  entailed  high 
risks  in  several  areas  including  the  gun- launcher  through  which  both 
conventional  ammunition  and  the  guided  missile  could  be  fired;  the 
anti-tank  guided  missile  itself;  the  combustible  cartridge  case  for  the 
conventional  ammunition  for  the  main  armament;  the  night-firing  system; 
and  the  making  of  a  tank-like  vehicle  which  could  swim  and  be  parachuted 
from  aircraft.  While  all  of  these  feature*  had  beeifthc  subject  of  previous 
VO  efforts,  none  of  them  had  been  proven  m  prior  operational  systems. 

By  early  19M,  the  first  pilot  models  (prototype)  had  been  assembled 
and  delivered  to  the  If,  S.  Army  Test  and  Evaluation  Command  (TECOM) 
for  Engineering  Test  (determination  as  to  whether  „!<c  manufacturer  ha* 
met  the  stated  specifications)  and  Service  Test  (determination  of  suit¬ 
ability  for  If,  S,  Army  use). 


The  Service  Test,  the  first  operational  test  to  which  a  new  system 
is  subjected,  revealed  ninety-eight  deficiencies!./  and  one  hundred  twenty- 
six  shortcomings?/  and  resulted  in  the  waiver  of  three  unachievable 
requirements  which  were  included  in  the  approved  military  characteristics 
for  the  system.  The  final  report  of  this  test  contained  a  conclusion  that 
the  M551  was  not  suitable  for  U.  S.  Army  use  until  the  deficiencies  and 
shortcomings  had  been  corrected.  While  the  Service  Test  was  underway, 
action  had  been  taken  to  type  classify  the  M551  for  limited  production, 
and  a  multi-vcar  production  contract  had  been  awarded.  The  system  was 
type  classified  Standard  A—^  in  May  1966,  approximately  six  months 
after  the  completion  of  the  Service  Test.  (Inclosure  1  presents  a  graphic 
relationship  between  tests  and  decisions.  )  This  testing  of  pilot  models 
continued  with  climatic  tests  in  desert  and  arctic  environments. 

In  late  1966,  five  early  production  M551s  were  delivered  to  the 
U.S.  Army  Test  and  Evaluation  Command's  Armor  and  Engineer  Board 
for  Confirmatory  Test.  During  this  test,  representative  crews  were 
exercised  in  a  simulated  combat  environment  in  a  manner  designed  to 
tt  it  all  components  of  the  M551.  Jhe  final  report  of  this  test  indicated 
that  of  the  ninety-eight  deficiencies  found  in  the  Service  Test,  forty-five 
had  not  been  corrected  in  the  production  models  and  that  of  one  hundred 
twenty-six  shortcomings  previously  iound,  forty-seven  had  not  been 
corrected.  This  Confirmatory  Test  additionally  revealed  one  hundred 


l_/  Deficiency  -  A  deficiency  nor  lally  disables  or  immobilises  the 
equipment;  and  if  occurring  during  test  phases,  will  serve  as  a  bar 
to  type  classification  action, 

2 /  Shortcoming  -  Will  not  cause  an  irr.mediat:  breakdown,  jeopardise 
safe  operation,  or  materially  reduce  the  useability  of  the  item.  If 
occurring  during  test  phases,  should  be  corrected  if  it  can  be  done 
without  unduly  complicating  the  item  or  inducing  another  undesirable 
characteristic  such  as  increased  cost,  etc. 

1/  Standard  A  -  A  preferred  and  fully  acceptable  item  which  has  success¬ 
fully  completed  all  required  test  and  evaluation,  meets  DA  approved 
requirements,  military  characteristics,  and  iipcc  ifications  for  world¬ 
wide  or  specified  geographic  areas,  is  totally  suitable  for  performing 
the  required  mission,  can  be  properly  maintained  and  logistioally 
supported  m  tile  area  or  environment  in  which  the  item  is  to  be  used, 
and  is  being  or  can  be  produced  m  quantity. 


fourteen  new  deficiencies  and  one  hundred  twenty-eight  new  shortcomings. 
This  test  report  also  indicated  that  of  the  eighty-two  requirements 
subjected  to  measurement  in  this  test  which  were  included  in  the  approved 
military  characteristics  for  the  system,  sixty-five  were  met  or  exceeded 
by  the  test  items.  The  most  important  area  of  failure  to  meet  the  criteria 
of  operational  suitability  was  the  conventional  ammunition. 

The  operational  testing  and  evaluation  of  this  system  continued  for 
the  ensuing  three  years  in  the  foim  of  climatic  tests,  check  tests, 
confirmatory  tests,  product  improvement  tests,  and  troop  tests.  Many 
of  these  later  tests  used  either  pilot  or  early  production  models  which 
had  not  yet  been  modified  to  correct  the  known  deficiencies  and  con¬ 
sequently  old  deficiencies  were  rediscovered  and  some  new  ones  were 
identified. 

Based  on  test  results,  various  modifications  to  the  system  have 
been  developed,  and  retrofit  programs  have  been  initiated  to  provide 
vehicles  which  are  suitable  for  U.  S.  Army  use. 

Modified  systems  have  been  deployed  to  some  U.  S.  Army  units 
in  the  continental  U.  S.  ,  Vietnam,  Europe  and  Korea.  The  CONUS  unit 
participated  in  a  U.  S.  Army  Combat  Developments  Command  Troop 
Test  of  A  Light  Armor  Battalion.  While  this  test  was  designed  to 
evaluate  the  organization  and  doctrine  for  the  test  unit,  there  were 
some  side  evaluations  of  M551  operational  suitability.  The'deficiencies 
revealed  by  this  test  had  been  previously  discovered  by  the  TECOM 
tests.  One  significant  modification  was  recommended  -  the  addition 
of  a  range  finder  for  use  when  firing  conventional  main  gun  ammunition. 
This  recommendation  was  based  on  an  unacceptably  low  percent  of 
first  round  hits  during  the  test.  Using  commands  in  both  Vietnam  and 
Europe  were  asked  to  conduct  evaluations  of  the  M551  when  initial 
deployments  were  completed.  The  deficiencies  found  in  each  theater 
had  been  identified  by  previous  TEiCOM  tests.  Coincidentally,  each 
theater  reinforced  the  CONUS  recommendation  concerning  adding  a 
range  finder.  Othei  modifications,  peculiar  to  the  environment  in 
each  theater  of  operations,  were  recommended. 

Analysis 

The  operational  testing  and  evaluation  of  the  M551  was  both 
thorough  and  complete.  In  fact,  the  test  program  could  be  more  properly 
faulted  for  overtestmg  than  for  undertesting. 
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The  TECOM  testing  was  designed  to  determine  the  degree  to  which 
the  system  satisfied  a  rigid  set  of  military  characteristics  which  were 
adopted  very  early  in  the  program  and  which  were  not  revised  to  keep 
abreast  of  rapidly  changing  operational  concepts  within  tiie  Army  -  the 
emergence  of  airmobile  operations  as  a  logical  successor  to  airborne 
operations.  Furthermore,  these  characteristics  were  not  challenged 
by  the  ultimate  users  as  a  result  of  their  operational  tests  and  evaluations. 

Many  of  the  design  characteristics  which  were  the  subject  of 
recommendations  for  modifications  were  dictated  by  the  requirement 
for  the  vehicle  to  be  air  droppable.  Relaxation  of  this  requirement 
very  early  in  the  development  would  have  permitted  the  developer 
greater  latitude  in  designing  against  truly  critical  requirements. 

It  appears  in  retrospect  that  too  many  high  risk  components  were 
included  in  this  one  system.  Testing  illustrated  this  fairly  early  in 
the  development  of  the  system  but  irreversible  commitments  had  been 
made  to  the  production  of  the  system  before  these  test  results  had  been 
given  sufficient  vlsioility  at  the  proper  levels. 

TEST  HISTORY  OF  THE  PERSHING  WEAPONS 
SYSTEM  AS  IT  RELATES  TO  MAJOR  DECISIONS 

Backg  ound 

The  PERSHING  Program  has  experienced  very  little  criticism  from 
Congress  or  the  General  Accounting  Office  when  compared  to  the  M551 
(SHERIDAN/SHILLELAGH)  weapons  system.  However,  in  many  respects, 
the  two  development  programs  were  very  similar.  Both  weapons  systems 
were  Type  Classified  Limited  Production  (TC-LP)  before  the  Operational 
Test  and  Evaluation  (i.  e.  ,  Service  Test)  was  completed.  In  fact,'  if  any¬ 
thing,  me  PERSHING  missile  tepc  program  was  more  compressed  with 
TC-LP  in  July  1961  over  two  years  prior  to  the  beginning  of  integrated 
Engineering  Test  and  Service  Test  (ET/ST)  (Sept  63  -  Jan  64).  This 
case  study  will  review  those  features  of  the  PERSHING  Program  which 
helped  it  to  become  a  relatively  successful  R&D  effort,  even  though 
production  decisions  were  made  two  years  before  the  required  Army 
testing  (EI/ST)  was  even  started.  (See  Inclosure  2.  ) 
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Discussion 


The  PERSHING  weapons  system  was  developed  as  a  smaller,  more 
mobile  replacement  for  the  REDSTONE  missile,  which  had  been  developed 
in  the  early  fifties  and  placed  in  the  field  m  1955.  The  initial  Qualitative 
Military  Requirement  (QMR)  was  approved  by  Department  of  Army  in 
July  1957.  At  first  the  range  was  not  specified:  however,  the  missile 
weight  was  limited  to  a  maximum  of  10,  000  lbs.  The  PERSHrNG  system 
was  designed  to  be  transported  on  an  XM474E1  tracked  vehicle  and  to 
be  helicopter  transportable,  feasibility  studies  were  conducted  for  the 
PERSHING  missile,  and  finally  on  7  January  1958,  the  Secretary  of 
Defense  authorized  the  Army  to  begin  its  development  program.  (See 
Inclosure  2.  ) 

The  Martin  Marietta  Corporation  at  Orlando,  Florida,  was 
awarded  the  contract  for  research  and  development  of  the  PERSHING 
on  28  March  1958.  A  second  generation  missile  system,  PERSHING  la, 
was  also  developed  by  Mai&n  beginning  in  mid-1964  to  fill  the  Quick 
Reaction  Alert  (QRA)  mission  in  Europe.  The  major  changes  were  to 
increase  the  number  of  launchers  from,  eight  per  battalion  to  thirty- six 
per  battalion.  All  tracked  vehicles  were  replaced  by  new,  high-mobility, 
wheeled  vehicles.  The  number  of  programmed  test  stations  was 
increased  from  four  to  twelve  per  battalion.  Essentially,  the  PERSHING 
la  is  a  product  improvement  of  the  PERSHING  I  missile  and  therefore 
war  a  relatively  low  risk  development  effort. 

The  development  approach  for  each  end  item  consisted  of  preparation 
of  detailed  component  specification,  component  design  and  test,  bread¬ 
board  fabrication  and  test,  engineering  model  fabrication  and  test, 
followed  by  prototype  fabrication  and  test.  The  end  items  then  were 
mtrried  into  a  system,  arid  a  weapon  system  test  program,  including  a 
flight  test  program,  was  conducted.  Finally,  a  rather  limited  Engineering 
Test  and  Service  Test  concluded  the  development  program. 

The  PERSHING  weapon  system  development  program  has  generally 
been  successful,  meeting  milestones  without  any  significant  increases  in 
cost  over  the  planned  expenditures.  This  success  has  been  obtained 
without  the  benefit  of  either  the  Engineering  Test  or  Service  Test  before 
a  decision  was  made  for  production.  In  fact,  the  PERSHING  I  was  issued 
to  missile  battalions  at  the  same  time  that  the  integrated  "’/ST  was  just 


beginning.  Yet,  the  PERSHING  I  system  met  most  of  the  characteristics 
specified  in  the  QMR  such  as  target  engagement,  transportability, 
environmental  effects,  configuration,  safety,  range,  accuracy  and 
rate  of  fire.  The  PERSHING  I  system  did  not  fully  meet  the  require¬ 
ments  for  reaction  time  and  reliability;  however,  the  PERSHING  la 
showed  improvement  in  both  these  areas. 

Analysis 

There  are  a  number  of  factors  which  enabled  the  PERSHING  missile 
program  to  achieve  success  tn  the  same  type  of  environment  that  caused 
the  M551  (SHERIDAN /SHILLELAGH)  to  do  so  poorly. 

t.  There  was  a  rather  extensive  Engineering  Design  Test 
Program  conducted  by  both  the  contractor  and  the  Army  Missile  Command. 
Deficiencies  were  identified  early,  and  corrective  action  taken  to  prevent 
reoccurrence. 

2.  A  highly  competent  team  was  formed  at  the  Project  Manager 
level  with  continuity  provided  by  key  civilian  personnel. 

3.  Project  Managers  have  been  highly  competent,  and  their 
replacement  has  been  conti oiled  to  occur  at  logical  break  points  in  the 
development  program. 

4.  There  was  a  solid  technical  base  a  Redstone  Arsenal 

which  could  draw  on  experience  obtained  from  t)  JUPITER  and  REDSTONE 
missile  program, 

5.  Adequate  funding  has  been  provided  for  the  development 
program  without  any  major  budget  cuts  which  mignt  have  affected 
development. 

6.  There  has  boon  strong  OSD  support  for  the  PERSHING  la 

Program. 

7.  Command  interest  at  all  levels,  including  overseas 
commanders,  has  existed  throughout  the  PERSHING  Program. 

8.  A  PERSHING  office  was  established  at  Department  of  the 
Army  level  which  aided  the  overall  management  of  the  program. 


9.  A  competent  technical  and  management  team  was 
assembled  by  the  Martin  Company  to  work  on  the  PERSHING  contract. 

10.  A  high  level  of  motivation  was  found  in  personnel  at  all 
levels  from  the  contractor  to  the  tactical  units.  There  was  a  determina¬ 
tion  to  make  t^e  system  work. 

11.  Most  important,  there  was  a  continuo.it  interface  between 
the  users,  the  developers  and  the  contractors.  Informal  channels  of 
communication  were  kept  open  between  all  echelons. 

In  summary,  the  OT&.-E  that  was  accomplished  during  the  develop¬ 
ment  cycle  was  more  in  terms  of  operational  evaluations  with  Engineer¬ 
ing  Design  Test  providing  most  of  the  test  data.  The  best  OT&E  was 
performed  in  the  missile  battalions  that  were  activated  using  the  pro¬ 
duction  missiles.  The  PERSHING  Piogram  demonstrates  clearly 
that  the  production  decision  can  be  made  before  the  OT&E  is  accomplished, 
providing  other  factors  are  present.  In  this  case,  the  strong,  centralized 
management  teams  formed  by  both  the  contractor  and  the  Army,  plus  the 
close  oordinatior.  between  the  user,  developer  and  contractor  personnel 
helped  to  keep  the  program  out  of  serious  trouble. 

TEST  HISTORY  O r  THE  OH -6 A  (CAYUSE) 

HELICO°TER  AS  IT  RE LaTES  TO  M AJOR  DECISIONS 

Background 

The  OH -67  helicopter  n?.s  been  selected,  to  illustrate  the  develop¬ 
ment  of  a  system  under  a  more  conservative  philosophy  than  that  used 
for  the  other  Army  systems  examined,  Sheridan  and  Pershing. 

Discussion 

The  recc  rement  for  a  Light  Observation  Helicopter  (LOH)  which 
was  to  rep!-.  .he  H  - 1  3,  11-23  and  L-  1 9  our  mg  the  period  1960-1970 
was  approved  in  May  I960.  This  requirement  called  for  a  lightweight, 
reliable,  easily  maintainable,  readily  air  transportable  helicopter 
capable  of  performing  the  following  missions:  visual  observation  ar.d 
target  acquisition  reconnaissance,  and  command  control.  The 
helicopter  was  to  be  of  minimum  size  consistent  with  the  requirement 
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for  a  pilot  and  three  pasrengcrs,  or  a  pilot  and  four  hundred  pounds  of 
cargo.  Reliability  and  front-line  supportabii.ty  were  to  be  given  primary 
consideration. 

Aircraft  manufacturer s  were  invited  to  submit  design  proposals 
for  the  LOH  in  Octobei  1960.  In  response  to  this  invitation,  twelve 
companies  submitted  a  total  of  seventeen  desigr  proposals  for  the  LOH 
in  January  1961. 

A  design  selection  board  was  convened  in  April  1961  to  evaluate 
the  proposals  and  select  two  or  more  designs  to  be  developed  through  the 
flying  prototype  stage.  Three  designs  were  ultimately  selected  lor 
development.  Two  of  these  designs  were  somewhat  conservative  pro¬ 
duct  improvements  of  existing  de.igns  while  the  third,  later  selected 
for  production,  was  an  innovative  dt  arture  from  past  light  helicopter 
design.  Ah  three  were  to  use  the  same  engine  which  was  being  developed 
separately  to  provide  an  extreme'y  lig’ t- weight  turbine  engine  for  light 
helicopter  application. 

In  October  1961  the  planned  engine  for  the  LOH  encountered 
developmental  difficulties  of  such  seriousi.e.s  that  development  was 
initiated  on  a  second,  back-up  engine.  This  back-up  program  was 
later  terminated  when  the  difficulties  with  the  primary  engine  were 
overcome. 

The  three  prototypes  (five  aircraft  of  each  model)  were  subjec  id 
f.o  an  intensified  military  potential  test  during  the  p  Tied  March  to 
June  19o4. 

A  design  selection  board  was  appointed  in  September  1964  to 
select  the  most  suitable  design  for  the  LOH  mission.  This  board, 
after  exhaustive  analysis  of  the  results  of  the  military  potential  test, 
recommended  a  price  competition  for  at  least  1,000  aircraft  be  con¬ 
ducted  between  the  two  designs  selected  as  most  nearly  meeting  the 
Army  requirement.  In  October  1964,  the  Secretary  of  the  Army 
approved  the  board  recommendation  but  reduced  the  quantity  to  714. 

The  01I-6A  was  typo  classified  tot  limited  production  iu  Septemoer 
1964  and  a  multi-year  production  contract  was  awarded  in  May  1965. 
Engineering  and  Service  Tests  were  completed  in  late  1965  and  tne 
system  was  type  classified  standard  A  in  July  1966. 
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t  Various  other  tests  and  cveiV-ticns  were  conducted  through 

tr.ld-1968  (see  Inclosure  ?.  fo r  a  graphic  presentation  of  test  history), 
and  units  Were  equipped  3s  early  as  December  1966. 

Analysis 


This  helicopter  followed  an  almost  idealized  development,  test 
and  evaluation  pattern.  The  development  of  turee  competitive 
prototypes  was  feasible  due  to  the  relatively  low  cost  involved.  The 
direct  competition  between  the  three  models  in  the  Military  Potential 
Test  gave  the  decision  makers  an  objective  assessment  of  the  options 
available  to  them. 

The  testing  of  prototypes  enabled  the  Army  to  discover  major 
ues. go  deficiencies  before  a  commitment  to  full  production  was 
incurred.  The  production  specifications  could  therefore  incorporate 
the  resists  of  tests  and  preclude  the  production  of  items  with  known 
deficiencies. 

The  deve’opinent  and  testing  of  several  prototypes  did  not  result 
in  increases  in  either  cost  or  lead  time.  The  initial  production  price 
was.  very  reasonable  and  the  first  units  were  equipped  with  a  fully 
tested  and  fully  operational  system  only  six  years  after  the  establish¬ 
ment  of  the  requirement  tor  the  system. 

The  timely  provision  of  test  results  to  the  decision  makers  permitted 
their  proper  consideration  before  decisions  were  made.  The  inclusion  of 
test  results  in  the  bases  for  decisions,  particularly  in  the  early  phases 
of  development,  resulted  in  the  production  of  a  system  which  was  not 
plagued  by  costly  retrofit  problems  after  the  system  was  fielded, 

TROOP  TF.ST  OK  LAND  NAVIGATION  SYSTEMS 


This  case  hiatory  la  being  presented  because  this  test  repres*  nts 
n  dcn.irture  from  past  tent  philosophy  mid  will  probably  be  a  precursor  of 
t<’Nt»  t«  come. 
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Background 


The  U.  S.  Arir.y  has  recognized  a  requirement  for  navigation 
aids  in  combat  ve  lictes  since  World  War  II.  Several  development 
prog-ams  have  been  undertaken  in  the  ensuing  years,  but  none  produced 
a  satisfactory  system  until  a  Canadian  company,  Aviation  Electric, 

Ltd.,  combined  the  results  of  pr.  .ions  efforts  with  newly  developed 
components  to  produce  a  workable  system  in  the  early  1960s.  This 
system  was  tested  by  the  U.  S.  ,  British,  and  Canadian  Armies  and 
various  improvements  were  suggested  as  a  lesult  of  phese  lists. 

The  system  which  emerged  was  considered  operationally  suitable  by 
the  Canadian  and  British  Armies  and  has  been  deployed  by  boi'-. 

Discussioi 

In  1967,  it  was  decided  by  tne  Department  of  the  Army  that  a 
Troop  Test  of  Land  Navigation  Systems  was  required.  This  test  was 
to  use  the  Canadian-developed  systems  in  their  then  current  configura¬ 
tion.  The  test  was  to  determine: 

1.  The  relative  tactical  advantage,  during  day  and  night 
operations,  gained  by  type  test  organizations  using  various  mixes  of 
land  navigation  systems; 

2.  The  relative  navigational  advantage  during  day  and 
night  of  type  test  platoons /battery  command  and  control  elements 
equipped  with  various  mixes  of  land  navigation  systems; 

3.  The  workability  of  proposed  type  test  organizations 
equipped  with  land  navigation  systems  and  proposed  test  training 
program  for  use  of  land  navigational  aids; 

4.  The  operability,  reliability,  and  maintainability  of 
the  land  navigation  systems;  and 

5.  The  logistic  and  maintenance  requirement:!  to  support 
units  equipped  wth  land  navigation  devices. 

To  support  this  test,  a  limited  prntureme 't  of  navigation  systems 
(17t)  was  made.  These  systems  arc  being  installed  in  various  mixes  in 
:ypieal  combat  units.  These  units  include  a  tank  battalion,  a  mechanized 


infantry  battalion,  an  arra«rcd  ca*.:.l’.}  squa^ioe  (  ),  a  s  If- propelled 
artillery  battalion,  and  a  brigade  headquarters.  .  *  each  battalion, 
one  company /battery  will  serve  as  a  base  unit  and  will  hive  no  naviga¬ 
tion  systems  assigned;  one  company  will  have  a  vir /  au-,;ere  allocation 
of  systems  the  third  will  have  a  rather  generous  alVcation  of  systems. 

Each  o‘  these  units  will  be  exercised  in  accordart  e  with  a  set 
.cenario,  ar.d  *he  results  will  be  compared  to  determine  the  advantage 
gained,  if  any,  by  the  addition  of  land  navigation  systems. 

This  test  will  be  conducted  during  April  1970‘at  Fort  Carson, 
Colorado.  ' 

Analysis 

Tni3  test  will  embody  several  features  which  have  been  recommended 
by  various. high  level  bodies  such  as  Congress  and  the  President's  Science 
Advisory  Committee.  It  will  provide  for  operational  test  and  evaluation 
of  the  navigation  systems  before  full  scale  procurement  is  initiated, 

Tne  system  is  a  fully  developed  foreign  product  which,  if  procured, 
will  be  produced  in  the  U.S.  The  test  will  establish  a  current  capa¬ 
bility  against  which  the  tested  capability  can  be  compared. 

TES:  HISTORY  OF  THE  A-6  A  (GRUMMAN  ALL  WEATHER 
ATTACK  AIRCRAFT!  AS  IT  RELATES  TO  MAJOR  DECISIONS 

background 

The  Operational  requirement  for  the  A-6  was  pr  imulgated  in 
October  1956,  A  contract  was  let  for  4  R&D  aircraft  in  1958  and  4 
in  1959.  The  first  flight  of  the  A-6  was  conducted  in  April  I960  and 
'lie  first  production  contract  for  12  A/C  was  let  in  July  of  I960  for 
delivery  at  the  rate  of  1  per  month,,  starting  in  January  1962.  (See 
Inclosurc  4). 

P'lHCUHSion 

Phase  I  of  the  Navy  Preliminary  Evaluation  (NPE)  was  completed 
on  30  Novcn. her  I960.  The  NPE  is  the.  Navy’s  first  chance  to  evaluate, 
with  Navy  pilots  Hying  the  aircraft,  .he  characteristics  and  suitability 
of  a  wi.apo’i  system  for  Navy  use.  The  p.irpoBc  was  to  evaluate  at  an 


i> 

n  :  I 

U.3 

D  1 


F-12 


t 


early  date,  handling  qualities  and  performance  tests  in  order  to 
estimate  the  degree  to  which  operational  requirements  would  be  met 
and  to  allow  early  correction  of  deficiencies.  The  results  were  as 
follows: 


1.  5  highly  desirable  character. sties  which  were  worthy 
of  special  mention. 

2.  13  items  for  which  correction  was  mandatory  for 
satisfactory  service  use.  (No  major  items) 

3.  28  items  which  it  would  br.  desirable  to  correct. 

4.  1  item  that  should  be  considered  in  any  future  design. 

Phase  II  of  the  NPE  was  completed  on  8  December  1961  in  which 
tlying  qualities  with  latest  changes,  air  refueling,  mission  suitability 
with  avionics  systems,  and  all  weather  capabilities  were  tested.  In 
this  series  of  tes;s  there  were  numerous  discrepancies  that  needed 
correcting  prior  iO  service  use,  the  majority  of  which  were  in  the 
avionics  system.  The  terrain  avoidance  and  the  main  radar  presenta¬ 
tion  were  unacceptable  and  alternate  systems  were  recommended. 

Phase  Ilf  -  1  of  the  NPE  tests  was  completed  on  5  September  19t'2. 
The  purpose  was  to  evaluate  flying  qualities,  the  automatic  flight  control 
system,  the  production  cockpit  display,  night  air  refueling  capability 
as  a  tanker  and  corrections  to  deficiencies  previously  reported  by 
previous  test  results,  The  results  showed  that  34  of  the  previous  dis¬ 
crepancies  had  been  corrected  satisfactorily.  There  were  10  uncorrectod 
or  new  deficiencies  which  wore  considered  mandatory  ptlor  to  service 
use,  There  were  40  deficiencies  which  were  considered  desirable  to 
correct.  The  NPE  Hoard  recommended  that  the  10  mandatory  deliciencies 
bo  corrected  prior  to  delivery  of  the  A-6  to  the  Navy  fur  further  testing 
and  that  the  desirable  corrections  bo  corrected  on  a  high  priority  banu.. 
The  Hoard  of  Inspections  and  Survey  trials  started  in  October  1962  and 
were  to  continue  until  June  1965  before  the  A/C  was  formerly  accepted  for 
service  use.  The  Currier  Suitability  demonstrations  were  completed  on 
19  November  1962.  The  A-6  did  well  on  the  carrier  suitability  demount  ra¬ 
tions  an:i  had  only  5  deficiencies  which  were  considered  mandatory  to 
correct.  These  were  all  fairly  miner  and  correctable. 
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Although  the  BIS  trials  were  not  complete  at  this  time,  the 
results  of  Phase  III  NPE  and  the  car  rier  suitability  demonstrations 
showed  no  major  airframe  problem  areas  and  a  follow-on  production 
contract  for  23  aircraft  was  let  in  November  1962  and  for  an  additional 
37  in  June  1963.  Service  acceptance  trials  on  the  A-6  continued 
through  January  196S.  During  these  testa  various  discrepancies 
were  found  and  corrected.  The  avionics  system  was  a  big  step  forward 
in  the  statc-cf-the-art  but  was  plagued  with  various  problems,  the 
greatest  of  which  was  reliability.  The  mean  time  between  failure 
(MBTF)  of  various  components  did  not  meet  guaranteed  requirements 
in  many  instances.  By  redesign  of  bits  and  pieces,  changes  and  modifica¬ 
tions  which  were  incorporated  on  production  line  A/C  as  soon  at  they 
proved  to  be  satisfactory,  performance  improved  greatly  and  In  March 
of  1965  the  President  of  the  Board  of  Inspection  and  Survey  recommended 
tnat  the  A-6  A  be  accepted  for  service  use  when  specified  mandatory 
deficiencies  were  corrected.  On  14  March  1966,  the  Chief,  Bureau  of 
Naval  Weapons  reported  to  the  Secretary  of  the  Navy  that  the  deficiencies 
had  been  corrected,  except  for  a  few  minor  exceptions  where  further 
development  was  still  going  on,  and  recommended  that  considering  the 
high  degree  that  the  A-6  A  met  the  contractual  requirements  that  it 
be  accepted  for  service  use. 

Analysis 

Because  of  the  press  of  .'EA  the  A-6  was  introduced  into  a  fleet 

squadron  in  November  1963  and  deployed  to  SEA  in  late  Spring  1965. 

•• 

!  The  Operational  Test  *r  Evaluation  Force  VX-5  received  its 

first  A/C  for  operational  test  and  evaluation  In  March  of  1965.  This 
was  because  spares  and  support  equipment  were  not  available  for  support 
of  another  site.  However,  OT6E  and  Replacement  Air  Croup  Training 
were  conducted  together  at  the  Naval  Air  Station,  Oceana  starting  in 
October  1962  concurrently  with  the  Service  Acceptance  Trials  at 
Patuxent  River.  This  its  not  desirable  because  the  fleet  had  to  train 
and  develop  tactics  with  aircraft  that  were  not  fully  tested  and  had 
various  deficiencies  to  overcome.  It  did,  however,  get  squadrons 
trained  to  deploy  with  a  capability  and  aircraft  that  were  sorely  needed 
in  SEA  in  the  Spring  of  1965.  The  aircraft  which  the  squadron  deployed 
with  were  right  off  the  production  line  with  all  of  the  changca  and 
modifications  incorporated  that  the  Board  of  Inspection  and  Survey 
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recommended  in  .ts  March  1965  report.  If  the  production  line  had 
been  closed,  after  delivery  of  20  A/C  in  1962,  until  testing  recommended 
service  acceptance  in  1965,  and  a  decision  to  produce  was  then  made, 
it  would  have  been  approximately  1968-70  before  the  A-6  could  have 
been  deployed  inbombat.  The  A-6  has  proved  to  be  an  outstanding 
weapon  system  which  performed  exceptionally  well  in  combat  in  SEA. 
This  tapabili'y  might  not  have  been  achieved  in  SEA  if  the  ''Try  before 
Buy"  concept  had  been  used. 

TESTING  HISTORY  OF  THE  C-141  AIRCRAFT 
(CATEGORIES  I.  II.  AND  III) 

Background 

The  C-141  aircrafr  has  entered  the  U.S.  Air  Force  inventory  as  a 
principal  item  in  the  attainment  of  a  new  capability  in  its  world-wide 
logistic  support  mission  (System  476L).  It  has  been  produced  in  quantity 
(284)  after  having  undergone  the  complete  acquisition  and  operational 


test  cycie  prescribed  ;n  Air  Force  Regulation  80-14. 


lirplane 


itself  features  four  ti  rbofan  engines  separately  mounted,  a  high  w:ng 
with  25°  sweep,  fus  ,-lage  mounted  landing  gear,  truck  bed  height  level 
cargo  compartmr.i.  and  high  T-tail  empennage.  Its  principal  peiformar.ee 
characteristics  a' e  as  follows  (approximately): 


Takeoff  weight,  maximum 
Pay'oad 


Speed,  maximum 

Average  cruise  speed 

Takeoff  distance  over  50' 
obstacle  (sea  level) 

Landing  distance  over  50' 
obstacle  (sea  level) 


316,600  lbs 

59,300  lbs 
at  4000  NM 

506  Kta. 

431  Kta. 

5, 660  ft. @  max 
gross  T.  O.  wt 

3,  870  ft  @ 

25  7,  500  lbs 
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The  C-141  program  from  conception  to  production  fiiish,  covered 
the  time  period  from  May  1960,  when  the  Soecific  Operational  Require¬ 
ment  was  published,  to  the  first  quarter  of  CY  1968.  At  this  writing 
all  testing  of  the  system  has  been  completed  except  some  phases  of 
tne  fatigue  test  which  a.-e  expected  to  be  finished  in  1973,  due  to  a  USAF 
decision  to  test  to  a  factor  of  4  in  lieu  of  2  life  times. 

The  C-141  System  management  concepts  were  directed  toward 
organizing  and  employing  various  functional  agencies,  each  of  which 
made  its  own  particular  contribution  to  the  overall  development  and 
acquisition  plan.  These  were. 

Air  Force  Systems  Command  (AFSC)  -  Development,  test, 
procurement,  production 

Air  Force  Logistics  Command  (AFLC)  -  Supply,  maintenance, 
transportation 

Air  Training  Command  (ATC)  -  Trainirg 

Military  Airlift  Command  (MAC)  -  Operations 

U.S.  Army  -  Adviser 

Lockheed  -  Georgia  -  Manufacturer 

Federal  Aviation  Agency  -  Certification  (of  aircraft  for 
potential  commercial  applications) 

U.S.  Corps  of  Engineers  -  Facility  construction 

By  the  application  of  centralized  management  principles  the  inputs  of 
these  functional  agencies  were  coordinated  and  used  in  the  conceptual, 
development,  test  and  production  phases  of  the  program.  This  philosophy 
produced  an  integrated  effort  which  recognized  all  facets  of  the  system 
acquisition  problem  and  gave  proper  weight  to  each  of  them.  The  extent 
to  which  this  was  carried  in  the  evaluation  cycle  is  reflected  in  the  fact 
that  the  Category  III  testing  was  done  by  a  Joint  Test  Force,  in  which  the 
Military  Airlift  Command  was  joined  by  AFSC,  AFLC,  ATC,  the  U.S. 
Army,  and  the  contractor,  with  a  record  of  outstanding  success.  The 
final  result  Was  an  airlift  system  which  was  responsive  to  the  needs  of 
all  users,  operators,  and  support  agencies. 
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TEST  DIRECTIVES 


Category  I  -  Air  Force  Regulation  80-14 

Category  IJ.  -  Aeronautical  Systems  Division  of  AFSC 

Category  III  -  Military  Airlift  Command  -  Test  Plan 

dated  January  1965  authorized  by  AFR  80-14 

TEST  ORGANIZATION 

Category  I  -  Lockheed  -  Georgia 

Category  II  -  Air  Force  Systems  Command  (with  Lockheed  assistance) 
Category  III  -  Military  Airlift  Command  (Joint  Test  Force  with 
AFSC  and  Lockheed  assistance) 

TESTING  PERIOD  (See  Inclosure  5) 

Static- Test  -  June  1963  -  June  1965 

Fatigue  Test  -  August  1963  -  factor  of  2  completed  December  1967 
factor  of  4  will  continue  through  1973 
Category  I  and  II  -  September  19b3  -  May  1965 
Category  IJJ  -  April  1965  -  July  1966 

TEST  ITEMS 


Category  1-5  aircraft 

Category  II  -  3  aircraft  (first  production) 

Category  III  -  Entire  operational  fleet  then  in  inventory 
(approx  122) 


TEST  SITES 


Category  I  -  Lockheed  facility  in  Georgia  and  Edwards  AFB,  Calif. 
Category  II  -  Flight  test  at  Air'Force  Flight  Test  Center, 

Edwards  AFB,  California  and  Ft.  Bragg,  N.  0. 
Environmental  tests  at  El  Centro,  California, 

Eglin  AFB  and  Alaska 

Category  III  -  Travis  AFB,  California  and  MAC'S  world-wide 
routes 
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Test  Result* 

I 

Although  the  Air  Force  was  an  active  participant  in  the  Category  I 
tests  of  the  C-141  with  an  eye  toward  early  evaluation  o.  the  airplane, 
it  was  Category  II  testing  which  was  more  productive  from  a  military 
standpoint.  The  prime  Category  I  objective  was  to  obtain  the  FAA  type 
certification.  In  general,  the  Category  II  tests  showed  the  C-141  A 
capable  of  fulfilling  the  Specific  Operational  Requirement  (for  a  large 
capacity,  high  speed  transport  with  long  range  and  an  air  drop  logistic 
capability).  2500  flight  hours  in  one  year  of  testing  showed  no  significant 
airframe  structural  deficiencies.  There  were  shown  to  be  some  undesirable 
aerodynamic  characteristics,  none  of  them  major.  347  recommendations 
for  minor  or  desirable  improvements  were  made  as  a  result  of  Category 
II.  Category  III  tests,  which  employed  a  considerable  number  of  the  MAC 
fleet  then  in  inventory,  were  based  on  a  much  larger  sample  size.  The 
principal  results  of  these  tests  reinforced  the  findings  of  Category  II, 
which  was  significant  ir.  the  light  of  the  fact  that  data  were  obtained  in 
tactically  Realistic  conditions.  Information  was  derived  from  logistics 
missions  flown  in  support  of  the  Southeast  Asia  effort  in  addition  to 
several  large-scale  exercises  in  other  world  areas,  fn  general,  the 
C-141  was  found  to  be  tactically  suitable  for  its  intended  role,  to  be 
supportable  from  a  lcgistics  and  maintenance  standpoint,  and  to  possess 
the  operating  characteristics  that  are  desirable  in  a  Tong  range,  fast, 
versatile  transport  of  its  type.  Several  minor  problem  areas  were 
uncovered  most  of  which  were  correctable  by  small  design  changes  or 
local  modification  programs.  Ir»  the  main,  however,  the  Category  III 
portion  of  the  test  program  bore  out  the  wisdom  of  the  earlier  decision 
to  produce  the  aircraft. 

Decisions  Made  as  Result  of  Testing 

As  a  result  of  a  System  Discrepancy  Report  (SDR)  System  used 
during  Category  II  and  III  testing  of  the  C-141,  a  total  of  760  problems 
was  identified,  of  which  140  resulted  in  Engineering  Change  Actions 
(auxiliary  power  unit  exhaust,  redesign  of  cargo  ramp,  etc. )  146  were 
taken  care  of  by  local  action  (in  tho  operating  activity)  and  the  balance 
distributed  between  "Quality  Control"  and  "No  Action  Warranted."  As 
obseived  before,  the  testing  had  no  influence  on  the  acquisition  decision. 
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Costs 

(Limited  to  contract  costs  -  do  not  include  gcvoi nment-furnished 
materia!  or  manpower. ) 

Category  I  -  $46.  530M  (4,008,000  manhours) 

Category  II  -  3.  060M  (80,  030  manhours)  estimated 

Category  III  •  2.  349M  (2,010  flying  hours)  estimated 

TESTING  HISTORY  OF  F-5  AIRCRAFT 


Background 


The  F-5A/F  was  designed  as  a  clear  air  mass  fighter,  intend*, 
to  perform'the  classic  functions  of  the  fighter  mission,  i.  e. ,  air 
superiority,  interdiction,  close  air  support,  and  aerial  reconnaissance. 
It  is  a  relatively  light,  very  maneuverable  supersonic  aircraft. 
Simplicity  of  operation  and  maintainability  were  stressed  when  the 
airplane  was  designed.  The  F-5A  is  a  twin-engined  single  place 
aircraft  equipped  with  two  M-39  (20mm)  cannons.  The  F-5B  has  a 
second  cockpit  in  place  of  the  guns.  The  RF-5A  is  the  photo  recon¬ 
naissance  version  of  the  F-S  series  (See  Inclosure  6). 


The  F-5  aircraft  was  specified  to  fill  the  requirements  of  the 
U.S.  Military  Assistance  Program  (MAP)  for  a  simple,  effective, 
easily  supportable  fighter  weapon  system.  Its  design  was  an  outgrowth 
of  that  of  the  T-38  trainer  used  for  several  years  by  the  USAF  Air 
Training  Command.  Low  unit  eo6t,  low  operating  cost,  light  weight, 
case  of  maintenance,  two-engine  safety  and  relatively  high  performance 
are  features  of  the  concept.  Principal  differences  between  the  trainee 
and  the  F-5  arc  the  modifications  to  the  wing  structure  to  gwe  it  an 
external  load-carrying  capability,  its  armament  equipment,  and  its 
low  footprint  pressure  (to  enable  it  to  operate  from  sod  or  austere 
fields). 

The  testing  cycle  of  the  F-5  was  unique  to  the  extent  that,  In 
addition  to  the  regular  USAF  Category  1,  If,  and  III  testa,  the  system 
was  evaluated  in  a  combat  environment  when  a  12-aircruft  squadron 
was  deployed  to  Southeast  Asia  in  late  1905,  As  project  SKOSill  TIGER, 
the  F-5  squadron  was  compared  with  F-100  and  F-4  equipped  organizations 
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in  the  theate-  in  an  attempt  to  g  ot  data  bearing  on  force  mix  decisions, 
etc.,  i. e. ,  soi tie  rates,  logistic  comparability,  maintenance  require¬ 
ments  and  the.Uke.  Hie  F-5  tactical  fighte.  can  be  said,  therefore, 
to  have  been  thoroughly  evaluated  -  from  its  performance  as  a  training 
airplane  to  its  demonstration  in  combat. 

Test  Directives 
i  - 


System  Program  Directive  -  Feb  64 

Category  I  test  plan  prepared  by  Norvair  Division,  Northrop  Corp. 
Category  II/IU  test  plan  prepared  by  AF  Flight  Test  Center  aud 
approved  by  System  Project  Office  -  Systems 
Division,  Aeronautical,  AF  Systems  Command 


SKOSHI  TIGER  test  plan  prepared  by  AFSC/TAC 


Test  Sites 


Category  I  -  Contractor  facilities  and  Edwards  AFE 
Category  II/III  -  Edwards  AFB,  California 

Environmental  Tests  -  EgHn  AFB,  Yuma,  Arizona  and  Eilson  AFB, 

Alaska 

Test  Organisation 
Primary 

Category  I  -  Contractor  -  Norvair  Division  of  the  Northrop 
Corporation 

Category  II/III  -  Air  Force  Systems  Command,  Air  Force 
Flight  Test  Center,  Edwards,  AFB,  Calif. 
Category  II/PI  -  Tactical  Air  Command 
SKOSHI  TIGER  -  10th  Fighter  Command  Squadron 

Support 


Aeronautical  Systems  Division,  AFSC,  Wright- Patterson  AFB, 
Ohio 

-  Managing  Agency  for  F-5  Program  and  CAT  II 
Adverse  Weather  Teat  Program 
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Air  Proving  Ground  Center  (Now  Armament  Development  and 
Test  Center).  Eglin  AF  B,  F  iida  •  Climatic 
Laboratory  support  of  CAT  'i  Pregram 
Air  Training  Command,  R-  -•olph  AFE,  Texas 
Personnel  training  for  ali  pluses 
Ai**  Force  Logistics  Command,  Wright- Patterson  AFR,  Ohio 
-x>gistic  support  for  all  phases 

Test  Item* 

At  the  peak  test  period  in  the  CAT  I/II/Ui  cycle  there  v/ere  13 
aircraft  in  tl  e  test  inventory  (one  N-lb6  -  the  forerunner  of  the  J*-5. 
two  YF-5A  preproduction  models,  iv/o  F-5R,  and  eight  F-5A.L 

SKOSHI  TIGER  was  conducted  with  a  12-aircraft  squadron  of  F-5 
aircraft,  one  of  which  was  lest  to  enemy  action. 


Test  Results  and  Resulting  Decisions 


In  general,  the  tests  indicated  to  the  testing  organizations  and 
the  Systems  Project  Office  that  the  F-5  A/B  weapon  system  fulfilled 
the  Specific  Operational  Requirement.  The  recommendations  made  in 
the  test  reports  were  mainly  to  correct  mino’"  design  deficiencies  or  to 
h:ghlight  improvements  thought  desirable  but  which  were  outs’de  the 
original  requirement.  Of  tha  227  .  ecommer.dacions  made,  131  ,ere 
accepted  and  implemented  without  change,  68  were  implemented  af-'-.r 
revision  and  the  balance  were  rejected.  There  were  62  Engineeri1'- 
Charge  Proposals  generated  by  the  recommencctions  of  which  Sf'  % 
were  approved. 


SKOSHI  TIGER  testing  of  the  aircraft  (not  including  the  sortie 
rate  potential,  maintenance  evaluation,  and  logistic  support? bility 
investigation)  was  limited  to  s«ioty-of-f  light  items  ard  flight  clearance 
of  certain  armament  not  previously  cleared.  SKOSHI  TIGER  combat 
experience  led  to  the  incorporation  of  the  following  on  all  J’-5A 
airplanes: 
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a.  Gunbay  screens  (to  eliminate  ingestion  of  spent 
cartridge  cases) 

b.  Increased  ruddei  authority  (to  improve  ground 
handling  characteristics) 

C.  Windscreen  rain  removal  system 

d.  Improved  design  of  the  gun/bombing  sight 


Test  Costs 


Following  are  estimated  contract  costs  and  do  not  include  government 
furnished  material  or  manpower  (bombs,  ammunitions,  rockets,  fuel,  and 
oil.  and  military  and  civilian  personnel  at  the  test  sites).  The  latter  are 
not  readily  available. 

Category  I  -  $7.  9M 
Category  II/III  -  $1.4M 
SKOSHI  TiGEH.  -  $2C0K 

DEVELOPMENT  HISTORY  OF  THE  M-16  RIFLE  A/ 


Background 

The  M-16  was  designed  in  1957  by  Mr.  Eugene  M.  Stoner, 

Armalite  Division,  Fairchild  Engine  and  Airplane  Corporation,  to 
meet  an  expressed  Army  requirement  for  a  lightweight,  high-velocity 
rifle  of  smaller  caliber  than  the  standard  7.62mm  NATO  caliber.  Colt 
bought  production  rights  for  the  new  rifle  in  1959. 

Development  and  test  of  the  rifle  did  not  go  through  the  usual, 
rigorous  design  competition  and  lengthy  competitive  testing  to  verify 
that  a  detailed  sot  of  requirements  had  been  met.  The  Commanding 
General  of  Continental  Army  Command  had  informally  requested  Mr. 
Stoner  to  examine  the  Army's  need  for  a  lightweight  rifle  in  1957. 

Mr.  Stoner  developed  a  weapon  design  patterned  after  the  NATO  7.62mm 
AR-10  rifle  but  in  5.56mm  caliber.  This  rifle  was  called  the  AR-15 
in  the  commercial  version,  later  receiving  the  military  designation  of 
XM16E1  until  standardised  as  the  M-16  for  the  Air  Force  and  as  the 
M-16A1  for  the  Army  and  Marine  Corps.  (The  M-16A1  has  a  manual 
bolt  closure  device  not  on  the  M-16. ) 

1/  !r>clo«"r',s  7  and  n  contain  a  chronology  of  events. 
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Mr.  Stoner  designed  the  All- 15  to  use  cartridges  containing  a 
commercial  propellant  called  Improved  Military  Rifle  (IMP.)  powder 
and  using  commercial  primers.  Quantity  production  of  this  propellant 
later  became  a  major  problem  because  of  the  high  rejection  rates 
necessary  owing  to  powder  lots  exceeding  chamber  pressure  limits. 

Army  policy  of  dual  production  sources  ted  to  introduction  of  different 
types  of  IMR  propellant  and  to  production  of  ball  propellant  with 
different  characteristics  and  performance. 

IMR  and  ball  propellant  powder  had  substantially  different  chamber 
pressures.  Rail  propellant  also  caused  greater  fouling.  Coll  encountered 
excessive  malfunctions  with  ball  propellant;  in  1965,  more  than  half  the 
production  weapons  exceeded  bolt  cyclic  rate  limits  with  ball  propellant. 
To  meet  Vietnam  weapon  requirements,  the  Army  permitted  Colt  to 
use  IMR  powder  for  acceptance  tests  even  though  both  IMR  powder  and 
ball  propellr.nt  powder  cartridges  were  issued  to  the  troops.. 

To  overcome  malfunctions  encountered  with  the  use  of  ball  pro¬ 
pellant,  the  M-16  was  redesigned  with  a  modified  buffer  to  take  up  part 
of  the  bolt  recoil  and  to  slow  it  down.  Then  the  rifle  malfunctioned 
with  IMR  powder. 
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Cleaning  equipment  was  found  to  be  inadequate  or  lacking.  There 
was  only  a  limited  issue  of  ramrods.  Improper  lubricants  were  employed 
in  the  field  in  an  attempt  to  cope  with  the  malfunctions.  Use  of  these 
lubricants  caused  additional  malfunctions  and  corrosion. 

Discussion 


Arma'.ite  Division  produced  several  AR-15s  in  1958  and  delivered 
them  to  Ft.  Benning  and  Aberdeen  Proving  Ground  for  test.  In  tests 
at  Ft.  Benning,  the  AR-I5  malfunctioned  considerably  fewer  times 
than  d'd  the  standard  M-14  rifle  against  which  it  was  fired.  The  results 
were:  AR-15  -6.1  malfunctions  per  100  (3578  rounds  fired);  M-14  -  16 
malfunctions  per  100  (2337  rounds  fired). 


Mr.  Stoner  made  several  minor  modifications  as  a  result  of  these 
tests  including  two-piece  handguard,  modified  safety,  larger  outside 
barrel  diameter  at  the  muzzle  and  buffer  assembly  modification.  These 
became  standard  in  the  AR-15. 
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Concurrent  with  the  brief  Army  tests  in  1958,  Mr.  Robert  W. 
Macdonald  of  Coopcr-Macdonald,  Inc. ,  who  had  a  sales  promotion 
contract  with  Fairchild,  demonstrated  the  AR-15  and  M-14  in  the 
Philippines,  Malaysia,  Indonesia,  Thailand,  Burma,  Italy  and  India. 

The  first  AR-15s  (25)  were  sold  to  Malaysia  01  December  5,  1959; 
the  next  AR-15s  (1250)  to  Indonesia  March  1 1,  I960;  and  ARPA 
ordered  1000  AR-15s  on  December  27,  1961  for  test  in  Vietnam.  The 
Air  Force  first  procured  the  AR-15  (8500)  in  May  1962  by  contract 
with  the  new  manufacturer,  Colt  Industries,  and  the  Army  first  procured 
a  small  quantity  (338)  in  October  1962  for  evaluation  and  test.  The  next 
major  military  procurement  was  in  1964  when  the  Army  bought  104,000 
M-16s  (85,000  for  the  Army  and  19,  000  for  the  Air  Force).  By  December 
31,  1967,  the  Army  procurement  rose  to  996,157  total  since  its  first 
procurement  in  1962.  Over  fifty  other  countries  had  also  purchased 
small  quantities  of  the  M-16  during  the  period  1964-1967.  On  April  19, 
1968,  "second-source"  selection  contracts  were  let  to  General  Motor* 
and  Harrington-Richardson  for  240,000  rifles  each. 

Malfunctions  were  alleged  to  have  Caused  a  number  of  deaths  in 
lombat.  This  unfavorable  publicity  resulted  in  intensive  Congressional 
investigation  in  1967  and  eventual  product  modification. 

Although  there  is  no  formal  procedure  for  the  development  of 
small  arms,  it  can  be  seen  that  development,  test,  evaluation,  and 
purchase  of  the  M-16  did  not  follow  the  norm.  Different  DOD  agencies 
had  ordered  and  independently  tested  small  quantities  of  the  weapon. 

In  retrospect,  it  appears  that  these  tests  were  overlapping,  uncoordinated, 
and  carried  on  in  some  cases  under  insufficiently  qualified  supervisors. 

In  particular,  these  tests  did  not  reveal  many  of  the  problem  areas  found 
in  later  intensive  operational  use. 

In  tracing  the  history  of  the  M-16  many  facets  appear  obvious 
today  which  were  not  recognized  at  the  time.  From  most  accounts, 
the  AR-15  as  tested  i :  the  US  and  in  Vietnam  during  the  period  1958- 
1964  performed  very  well.  It  must  be  iccognized  that  these  tests  were 
with  a  different  vorsion  of  the  rifle  Ilian  that  which  malfunctioned  so 
frequently  in  the  period  after  1964,  the  M-16, 

On  the  other  hand,  the  tempo  of  the  Vietnam  war  was  different 
before  1965  when  there  was  a  maximum  of  17,000  US  troops  in  all  of 
Vietnam.  The  Vietnamese  soldiers  were  enthusiastic  about  the  1000 
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AR-15s  they  tested  over  an  extended  period  from  1962-1964.  Yet 
there  was  little  sustained  close  range  fighting  during  this  period. 

Air  Force  personnel  receiving  the  AR-15  after  1962  were  generally 
using  the  weapon  for  guard  duty  where  malfunctions  were  not  likely 
either  to  be  encountered  or  considered  quite  so  serious  as  in  close 
combat. 

After  1964,  the  situation  changed.  The  serious  sustained  close 
combat  by  many  units  of  the  several  hundred  thousand  troops  now  in 
Vietnam  generated  frequent  situations  where  life  was  dependent  upon 
weapon  performance.  Malfunctions  were  reported  in  increasing 
numbers. 

Later  investigations,  tests  and  inspections  have  shown  a  number 
of  causes.  These  include: 

o  Poor  troop  discipline  in  maintaining  the  rifles  ar.d  lack 
of  cleaning  equipment,  including  ramrods.  Without  ramrods,  there 
was  no  way  to  remove  a  shell  in  the  chamber. 

o  Redesign  to  accommodate  the  new  propellants.  This  caused 
malfunction  with  the  original  propellant  (IMR)  in  the  cartridges 
already  procured. 

u  Cartridge  magazines  had  springs  with  improper  tension, 
causing  jams. 

o  The  test  anc'  evaluation  of  different  lubricants  by  different 
combat  units  not  understanding  the  chemistry  of  the  lubricants  caused 
additional  malfunctions.  Improper  cleaning  and  lack  of  cleaning  equip¬ 
ment  also  resulted  in  many  weapon  failures. 

o  Unfortunately,  it  was  not  realised  for  several  years  that 
variations  in  powder  would  have  a  major  impact  on  M-16  functioning. 
Modifications  (made  to  correct  four  malfunctions)  caused  other 
malfunctions  when  powder  types  were  used  interchangeably.  The  IMR 
4475  propellant  gave  excessive  chamber  pressures  unless  there  was 
good  quality  control.  Ball  propellant  caused  moi1:  fouling.  It  also 
increased  the  cyclic  rate  of  the  bolt  from  the  norm  of  650-850  tc> 
over  1000  cycles  a  minute  and  this  caused  rapid  wear  on  parts  and 
cject'on  malfunctions.  Chrome  plating  the  chamber  added  for  hardness 
and  cleaning  case  reduced  malfunctions  with  IMR  but  aggravated  them 
with  bail  propellant. 
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The  M-16  malfunctions  became  increasingly  serious  after  1964 
as  major  weapon  procurement  was  increased  to  keep  pace  with  the 
risirg  rroap  strength  and  war  tempo  in  Vietnam. 

For  some  time  the  military,  OSD  and  contractor  efforts  to 
correct  deficiencies  appear  to  have  been  more  a  piecemeal  than  a 
systems  approach.  For  exa  e.  Col.  was  allowed  to  test-qualify 
production  weapons  using  the  iAlR  type  of  powder  that  worked  well 
although  the  weapons  would  malfunction  with  ball  propellant  also 
issued  to  the  troops. 

There  have  been  product  improvement  modifications  on  over  54 
of  the  223  pa'ts  of  the  original  AR-15,  commencing  with  minor  modifi¬ 
cations  by  Mr.  Stoner  after  hi3  1959  tests  to  the'  Army.  Through  mid- 
196V  there  were  159  engineering  changes  in  the  Colt  contracts,  of  which 
10  were  considered  by  the  program  manager  to  significantly  improve 
the  weapon. 

The  rifling  twi  Jt  started  at  1  to  14  (one  turn  of  the  bullet  in 
14  inches  of  travel)  but  was  changed  to  1  to  12  to  overcome  alleged 
instability  in  extremely  cold  weather.  Later,  the  original  twist  was 
tested  again  because  of  continued  uncertainty  of  performance  in  each 
case.  The  Army  also  added  a  manual  assist  bolt  closure  device 
requirement  although  testing  did  not  substantiate  a  need  for  it. 

In  1967-1968,  after  the  Ichord  Committee  had  reported  to 
the  public  on  the  M-16  rifle  problems,  the  Weapons  Systems  Evaluation 
Group  supervised  an  operational  test  in  Panama  under  simulated  combat 
conditions,  using  USMC  riflemen.  WSEG  conclusions  confirmed  the 
M-lo  variation  in  performance  with  different  modifications  and  different 
powers. 

The  cumulative  effect  of  Congressional  attention  and  DOD  investi¬ 
gation  finally  led  to  an  improved  M-16  variation. 

Conclusions 

Even  in  the  relatively  simple  M-16  rifle  system,  a  design  change 
to  one  portion  of  the  uystem  drastically  altered  the  performance  of 
other  elements  and  caused  serious  combat  malfunctions.  A  change  in 
one  part  of  the  system  should  have  been  fully  tested  with  the  system 
ar  a  whole  before  being  accepted.  _ 

Reproduced  from 
best  available  copy. 
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The  results  of  field  tests  of  proposed  changes  by  the  user  should 
have  been  treated  as  suspect  and  should  have  been  repeated  under 
controlled  conditions  Ly  qua’ Tied  test  personnel  for  verification. 

There  was  a  need  for  closer  Government  supervision  of 
contractor  production  practices  and  inspection. 

High-level  decisions  on  weapon  changes  were  made  without 
supporting  facts  and  were  unnecessary. 

The  Government  failed  to  plan  from  the  beginning  for  a  technical 
data  package  ar.d  manufacturing  rights  and  thus  paid  a  great  deal  in 
the  end. 

Poor  command  supervision  at  ail  levels  over  weapon  maintenance 
(inadequate  cleaning  and  care  of  weapons  in  the  field)  and  over  issuance 
of  cleaning  instructions  and  cleaning  materials  was  a  major  factor  in 
weapon  performance. 
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M551  WEAPONS  SYSTEM 

(SHERIDAN/SHIILELAGH) 
CHRONOLOGY  OF  TESTING  AND  DECISIONS 


—-xr  y.-*, 

* 

PERSHING !  ANi; 

CHRONOLOGY  OF  1 

1956 

1957 

1958 

1959 

1  I960 

•961 

1  1962 

Management 

Milestone! 


Production 

Milestone: 


Units 

Equioped 


Monogement 

Mile.*on*s 


Production 

Milestones 


Units 

Equipped 


I  PNC-ltsISF#  DfllrtN  tfata 


MHG  !  AND  fa  MISSILE  SYSTEM 
LOGY  OF  TESTING  AND  DECISIONS 


I9i2 

1963 

1964 

1965 

1966 

1567 

1968 

IV69 

1970  . 

1971 

197? 

■ 

— 

RING  TEST 

1 

ic5« 

__ 

“ 

fcSIGN  ItSI 

mi 

i  r 

1 

3ROINAIEO 

S 

UCTION 

RNCE 

• 

"  ' 

J  CON  TRAC 

r 

LIVERY  8EGI 

r 

NS 

PROOUCTIO 

A 

N  ENGINEER 

NG  ENDS 

PRODUCTI 

014  ENDS 

’ 

6 

1 

4 

i  i 

ATTALIONS. 

IQUIPPEO 

• 

• 

SERVICE  TE 

H 

38! 

SffisHSi 

tINC  TEST 
PRODUCTtCr1 

!  TEST 

QUICK 

A 

REACTION  A 

FIT  APFROV 

Ci 

< 

C 

CED  PRODUC 

IlOM  ENG  II 
iONIRACT  " 

A 

IK  FLIGHT 

A 

JEERING 

IYPE  CLASS 

flED  STAND/ 

>RD  "A" 

ENG! 

A 

NEERING  DE 

m 

ODUCTION 

V 

EST 

CONTRA 

:t 

■  ,  ■ 

_ 

1 

_ rfB 

L...  .  - - 

6TTALION5 

_ 

QUIPPED 

_ 

F-29  Inclorur*  i 


F-3U 


Inclosure  3 


MASTER  PHASING  PLAN 
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OPERATIONAL  TEST  AND  EVALUATION  RANGES/FACILITIES 


INTRODUCTION 


This  Appendix  presents  the  rest. Its  ol  the  OT&E  Task  Group's 
investigation  of  OT&E  ranges/facilities.  This  investigation  was  limited 
in  scope.  A  comprehensive  study  of  the  subject  would  require  time  and 
resources  far  beyond  what  was  available  to  the  Task  Group.  However, 
the  investigation  did  produce  much  pertinent  information  which  was 
useful  to  the  overall  OT&E  study.  A  summary  of  these  findings  on 
OT&E  ranges /facilities  appears  above  (pp.  34-36). 

The  primary  source  of  information  consisted  of  interviews  with 
personnel  involved  with  OT&E  both  within  the  Services  at  all  levels 
and  within  the  Office  of  the  Secretary  of  Defense.  Additional  informa¬ 
tion  and  data  were  obtained  from  visits  to  several  selected  OT&E 
facilities  in  the  western  part  of  the  United  Slates. 

RESULTS  OF  INTERVIEWS 

:•  If  appropriate  to  their  experience  and  function,  personnel  inter¬ 
viewed  by  the  Task  Group  were  asked  the  question: 

What  are  your  thoughts  on  test  ranges  and  their  control  - 
national  ranges,  Service  ranges,  contractor  ranges, 
methods  of  funding,  methods  of  scheduling? 

Summaries  of  interview  responses  to  ten  key  issue  questions  are 
provided  in  Appendix  E  to  this  report.  That  portion  summarizing 
responses  to  the  question  on  OT&E  ranges  is  extracted  and  included 
here. 


"Responses  to  this  question  varied  s^niewhat  according 
to  the  Service  affiliation  of  the  interviewee.  In  general,  U.  S. 
Army  personnel  considered  their  facilities  adequate  for  OT&E 
.  .  .  as  did  U.S,  Navy  personnel,  U.S.  Air  Force  responses 
indicated  general  satisfaction  with  R&D  test  fat  litics/capa- 
bilities  but  also  cited  OT&E  ranges  as  limited  in  proper  scope 
and  instrumentation.  In  this  regard,  the  Air  Force  proposed 
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concept  for  an  extensive  capability  to  conduct  integrated 
offensive-defensive  OTfiE  with  sizable  numbers  of  airborne 
systems  in  conjunction  with  realistic  simulated  threat  situations 
(the  HAVE  EDGE  study)  was  the  subject  of  some  comment. 
Although  such  a  capability  was  considered  ideal  and  certainly 
of  far-reaching  potential  for  needed  mission-oriented  OT&E, 
not  all  agreed  that  the  concept  was  feasible  within  the  present 
state-of-the-art  of  instrumentation  or  ability  to  construct  an 
inclusive  and  realistically  flexible  threat  environment. 

"A  serious  problem  of  concern  to  most  interviewees, 
regardless  of  Service,  is  the  slow  but  continuing  encroach¬ 
ment  on  range  space  and  facilities  bv  civilian  communities, 
civil  air  desires,  natural  resource  exploitation  interests, 
and  other  interest  groups  outside  of  the  Government.  Many 
ranges  have  lost  some  flexibility  to  do  comprehensive  OTfcE 
due  to  flight  restrictions. 

"In  general,  scheduling  of  Service  ranges  is  not  considered 
a  major  problem.  Each  Service  feels  its  ability  to  control 
its  ranges  is  essential  for  responsiveness.  In  addition, 
utilization  of  other  Service  ranges  when  necessary  is  not 
considered  a  major  problem.  Scheduling  and  reimbursement 
procedures  are  adequate  and  reasonably  responsive. 

"The  need  for  national  range  resources  is  generally 
appreciated;  however,  these  should  be  limited  to  those  unique 
and  costly  resources  of  common  use  to  all  Service  needs. 

"Funding  of  ranges  was  the  subject  of  extensive  comment. 
Funding  methods  and  procedures  vary  between  and  within  the 
Services.  Most  felt  that  the  Services  should  have  the  same 
type  funding  procedures.  The  majority  opinion  was  for  some 
level  of  effort  Service  funding  as  the  best  meat  s  of  preventing 
loss  of  capability.  However,  there  was  some  support  for 
industrial  funding  as  the  best  way  to  force  planning  and 
eliminate  inefficiencies. 


''Separate  funding  for  OTfrE  (as  a  program  element) 
was  given  some  support.  Identification  of  C/TfcE  fund  require¬ 
ments  by  project  was  considered  beneficial  -  particularly 
when  the  project  required  new  facilities /capabilities.  " 

ACTIVITIES  WITHIN  OSP  CONCERNED  WITH  OT&E  RANGES/FACILITIES 

At  the  OSD  level,  there  is  an  office  concerned  with  DOD  ranges: 
the  Office  of  the  Assistant  Director  (Ranges  and  Space  Ground  Support), 
Deputy  Director  (Strategic  and  Space  Sys.ems),  ODDRfcE.  This  is  a 
relatively  small  office  with  only  five  professionals;  however,  it  is 
responsible  for: 

Reviewing  and  recommencing  policies,  plans  and  programs 
pertaining  to  the  ground  environment  support  of  national 
space  and  missile  programs  to  include  - 

Development,  procurement,  alloca'ion,  management 
and  operation  of  missile  range  and  space  tracking 
networu  instrumentation; 

Coordination  of  NASA  and  DOD  range  instrumentation 
,  and  facility  plans,  and  ground  environment  support. 

Monitors  the  programs  and  budgets  of  the  National 
Missile  Range  a. 

This  office  is  primarily  concerned  with  National  Ranges  although  it 
maintains  cognizance  of  many  Service  range3/capabilities  to  the 
extent  possible  with  its  limited  manpower,  li  should  he  noted  that 
this  offi  e  is  under  a  separate  Deputy  Director  (Strategic  and  Space 
Systems)  from  the  Office  of  the  Assistant  Director  (Opcrat  anal  Test 
and  Evaluation),  which  is  under  the  Deputy  Director  (Administration, 
Evaluation  and  Management),  This  orgr  rizrtional  separation  introduces 
inevitable  difficult'es  of  communication  and  effective  cooperation. 

The  Ranges  and  Space  Ground  Support  Directorate  docs  participate 
in  meetings  of  the  Range  Commanders'  Council  and  its  various  subgroups. 
The  Range  Commanders'  Council  is  an  unofficial  body  that  arranges 
informally  for  dissemination  of  information  in  areas  of  mutual  interest 
to  the  technical  and  working  levels.  This  informal  activity  has  had  a 
beneficial  effect  on  facilities,  for  example,  the  Inter-Range  Instrumenta¬ 
tion  Group  (IRIG)  is  recognized  as  an  outstanding  body  in  the  field  of 
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range  tracking  management.  The  IRIG  ha*  established  telemetry 
standards  that  have  been  accepted  by  both  Government  and  industry. 
Membership  in  the  IRIG  is  not  limited  to  National  Ranges;  many 
Service  activities  participate.  House  Report  No.  1340,  ''Missile 
and  Space  Ground  Support  Operation",  by  the  Committee  on  Govern¬ 
ment  Operations,  cited  the  Council  and  its  subgroups  as  producing 
work 


".  .  .  characterised  by  objective  and  thorough  exploration 
of  matters.  Ry  contrast,  the  formal  coordinating  bodies 
frequently  tackle  a  problem  only  after  positions  are 
hardened,  and  solutions  some  about  largely  through 
time-consuming  bargaining  and  compromise  ..." 

Other  OSD  activities  have  been  involved  with  aspects  of  range/ 
facilities  documentation.  For  example,  OASD  (Installations  and 
Logistics)  has  developed  a  computerized  data  base  on  existing  ranges, 
with  pertinent  information  on  their  current  facilities,  capabilities, 
and  utilization.  A  first  print-out  of  this  data  base  was  completed 
in  June  1969.  Although  the  document  shows  potential,  it  is  incomplete 
in  its  present  form.  The  OT&E  Task  Group  believes  that  this  data 
base  should  be  expanded,  hopefully  in  a  more  useful  format,  and 
updated  at  regular  intervals. 

It  is  evident  that  OSD  attention  to  ranges  and  facilities  is  of 
a  fragmented  and  incomplete  nature.  There  seems  to  be  little  common 
diiection  or  guidance  provided  to  the  interested  parties  (the  National 
Range  focal  point,  the  I&Ldata  base  activity,  and  the  OT&E  function 
in  ODDR&E).  As  far  as  ranges/facilitics  are  concerned,  the  emphasis 
at  the  OSD  level  is  certainly  on  National  Ranges.  Essentially,  this 
means  emphasis  on  missiles  and  space  systems,  since  there  arc  the 
primary  areas  of  responsibility  for  the  National  Ranges.  As  a  result, 
little  icgular  attention  is  paid  to  tactical  OT&E  rangcs/capabilities 
(aside  from  missiles  that  relate  to  tactical  missions).  Although  the 
Services  bear  the  responsibility  for  developing  range  capabilities 
adequate  for  supporting  OT&E  of  their  tactical  systems,  Service 
efforts  in  this  regard  might  be  assisted  and  improved  by  greater 
emphasis  on  tactical  range  matters  at  the  OSD  level.  In  particular, 
an  OSD  organization  could  act  ^s  a  knowledgeable  advocate  of  legitimate 
Service  needs  and  provide  high-level  support  when  Service  ranges  are 
in  jeopardy  due  to  encroachment  by  non-  Defense  interests. 
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VISITS  TO  SELECTED  RANGE  ACTIVITIES 


Three  military  members  of  the  OT&E  Task  Group  visited  several 
ranges  in  order  to  obtain  firsthand  knowledge  of  current  capabilities, 
limitations,  and  problems.  .Specific  purposes  of  the  visits  were  to 
obtain  information  concerning: 

1.  CTSeE  ranges  a -<<J  facilit  ie  ,  including  scheduling  and 
funding  of  tests. 

2.  Data  collection  at  d  processing. 

3.  Analysis  and  evaluation. 

The  following  facilities  were  visited: 

Yuma  Proving  Ground  Command,  Yuma,  Arizona  (U. S.  Army) 
Naval  Weapons  Center,  China  Lake,  California  (U.  S.  Navy) 
USAF  Tactical  Fighter  Weapons  Center,  Nellis  Air  Force 
Base,  Nevada  (U.  S'  Air  Force) 

Pacific  Missile  Range,  C'::nt  Mugu,  California  (National 
Range  with  U.S,  Ni'vy  as  Executive  Agent) 

In  addition  to  those  involved  in  rnana ,  ;ng  these  facilities,  some  testing 
organizations  were  also  visited.  These  were  the  Navy  Test  Squadrons 
VX-5  (China  Lake)  and  VX-4  (Point  Mugu).I^ 

Some  general  observations  based  on  the  trip  as  well  as  some 
comments  on  individual  facilities  are  given  below. 


y  Relation  of  the  VX  Squadrons  to  U.S.  Navy  testing  is  covered  in 
Appendix  C,  "Operational  Test  and  Evaluation  in  the  Navy  and 
Marino  Corps". 


GENERAL  OBSERVATIONS 


Range  Facilities 

The  various  ranges  had  a  great  deal  of  capability  for 
hardware  item  testing.  There  was  some  overlap  and  duplication; 
however,  it  did  not  appear  to  be  unreasonable  and  was  perhaps  less 
than  might  have  btt-n  expected. 

There  was  much  less  capability  for  performing  tests 
aimed  at  determining  mission  and  combat  effectiveness.  This  was 
primarily  because  of  difficulties  inherent  in  quantifying  the  differential 
effects  of  combat  variables.  It  was  also  evident  that  there  is  currently 
less  emphasis  on  this  type  of  testing  than  would  be  desirable. 

Ballistic  missile  testing  was  a  case  where  't  is  virtually 
impossible  to  test  in  an  environment  which  simulates  combat  employment, 
including  enemy  capabilities  and  reactions.  The  Navy  is  somewhat  better 
off  in  that  they  can  launch  from  any  water  position  that  does  not  require 
overflight  of  land  areas.  The  Air  Force  is  presently  restricted  to  firing 
from  Vandenberg  Air  Force  Base.  Although  various  improvements  are 
being  made  to  increase  the  degree  of  realism,  the  most  valuable  step  - 
iiring  from  operational  sites  -  has  not  yet  been  approved. 

Funding 

This  is  an  area  where  there  are  as  many  ways  of  operating 
as  there  are  organizations.  Some  facilities  find  it  difficult  to  achieve 
the  proper  balance  between  workload  and  capabilities  and  prefer  a 
level-of-effort  type  funding  (e.g.,  Yuma  Proving  Ground).  Some  like 
industrial  funding  since  it  reveals  inefficiencies  and  forces  planning 
(e.g.  .  Naval  Weapons  Center).  Other  organizations  believe  that  industrial 
funding  would  result  in  less  and  less  utilization  which  might  in  turn  cause 
deterioration  and  loss  of  capability  (e.g..  Pacific  Missile  Range). 

Funding  is  an  important  problem  area  that  needs  high-level 
study  and  policy  guidance.  It  is  difficult  to  get  a  handle  on  the  funding. 
Because  each  organization  appears  to  do  it  differently  and  to  have  unique 
problems  in  computing  overhead,  etc.,  it  is  hard  to  make  meaningful 
comparisons.  The  most  attractive  method  of  funding  seems  to  be  the 
way  the  National  Ranges  are  now  funded,  rather  than  using  industrial 
funding  across  the  board. 
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A  Possible  New  Range  Activity  at  OSD 


Service  personnel  almost  unanimously  considered  this  of 
doubtful  value  and/or  potential.  It  v/as  obvious,  however,  that  they 
were  particularly  concerned  that  such  an  activity  migh*  become 
involved  with  control  or  direction  of  tests.  There  was  general  agree¬ 
ment  that  an  OSD  range  activity  could  help  to  obtain  required  resources 
far  the  Ser-.-'ros  and  to  provide  high-level  DO D  support  against  loss  of 
range  capabilities.  The  present  informal  set-up  under  the  Range 
Commanders'  Council,  1K1G,  and  other  committees  was  generally 
felt  to  be  effective  in  arranging  for  dissemination  of  timely  informa¬ 
tion  to  the  technical  and  working  levels. 

At  all  ranges  visited,  supervisory  personnel  stressed  th't 
scheduling  must  be  done  at  the  local  level  if  it  is  to  be  effective. 

No  range  reported  any  serious  inter-Service  problems  with  scheduling. 

•  •  •' 

The  following  sections  present  certain  selected  information 
on  the  ranges/facilitics  visited  in  the  course  of  this  investigation. 
Emphasis  is  on  significant  capabilities  a. id  limitations  which  have 
general  application  to  ranges /facilities  suitable  for  supporting  OT4.-E. 

YUMA  PROVING  GROUND.  YUMA.  ARIZONA  iU.S.  ARMY) 


Yuma  Proving  Ground  covers  about  1,  100  square  miles  of  desert 
land  in  the  southwest  corner  of  Arizona.  It  is  the  Army's  desert 
environmental  test  center  and  performs  both  ei  jineering  and  production 
type  tests  on  automotive  equipment,  munitions  and  weapons,  and  air 
delivery  materiel. 

Comments 

(1)  Range  facilities  appeared  to  be  excellent. 

(2)  Range  facilities  are  used  primarily  for  engineering  and 
production  type  tests  under  desert  conditions  wit),  a  minimum  of 
purely  OT&E  type  testing  effort.  Some  Army  preliminary  evaluations 
arc  being  conducted  on  proposed  weapon  systems  tor  the  Cheyenne. 
Yuma  does  not  simulate  hostile  environments  by  using  ECM,  since  this 
type  of  equipment  is  available  at  Fort  Htiachuca. 
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(3)  Scheduling  of  tests  presents  no  significant  problems. 
Normally,  a  customer  submits  a  request  for  tests  through  TECOM 
and  is  assigned  a  priority  cod.ng  I  to  V!.  Suitability  tests  on  develop¬ 
mental  materiel  are  scheduled  by  TECOM  in  a  similar  manner.  Daily/ 
weekly  test  schedules  are  developed  locally,  based  on  priority  and 
equipment  availability.  The  Marine  Corps  utilizes  Yuma  Proving 
Ground  tor  testing  landing  vehicles  and  weapons.  The  Navy  has  a 
permanent  detachment  stationed  at  Yuma  to  man  a  tracking  station 

for  them.  Yuma  works  with  China  Lake  and  other  Service  testing 
facilit'es.  For  example,  Yur.a  has  run  tests  on  a  !6-inch  gun  and 
the  Bullpup,  both  for  the  Navy. 

(4)  Funding:  Thirty  per  cent  comes  from  R&D  direct  funding 
to  Yuma  Proving  Ground;  the  rest  of  the  funds  come  from  customer- 
funded  test  programs. 

(a)  Yuma  prefers  levcl-of-effort  type  funding  rather 
than  industrial  or  Service  funding.  In  their  opinion,  the  current 
accounting  system  is  complicated  enough  and  industrial  funding  would 
make  it  even  worse. 

(b)  Current  funding  procedures  make  it  difficult  to 
keep  a  proper  balance  between  workload  and  capabilities  at  Yuma. 

About  one-third  of  their  funds  are  provided  in  the  RDTfcE  budget. 

The  remainder  fluctuate  in  a  random  fashion  due  to  customer  test 
programs.  For  example,  the  3d  Aviation  Company  now  stationed 
at  Yuma  provides  funds  for  about  twenty-five  per  cent  of  the  overall 
base  operation  costs.  When  this  Company  departs  about  30  June  1970, 
Yuma  will  have  a  funding  crisis  until  some  way  is  found  to  make  up  this 
deficit.  If  no  way  is  found,  there  may  be  a  considerable  reduction  in 
force, 

(5)  Integrated  Range  Instrumentation  Group  (1RIG):  The 
Yuma  Proving  Ground  participates  in  meetings  of  the  IRIG  and  feels 
that  it  is  the  best  way  to  coordinate  activities  and  efforts  among  the 
three  Services.  Formal  supervision  by  OSD  or  the  Army  is  not  con¬ 
sidered  advantageous  in  this  area. 


NAVAL  WEAPONS  CENTER,  CHINA  LAKE,  CALIFORNIA  (U.S.  NAVY) 


The  Naval  Weapons  Center  at  China  Lake,  California  (approximately 
155  miles  northeast  of  Los  Angeles,  California)  is  an  installation  of 
the  Naval  Material  Command  and  comprises  the  Naval  Weapons  Center, 
China  Lake,  and  the  Naval  Weapons  Center  Corona  Laboratories  (at 
another  Southern  California  location).  The  NWC  is  engaged  in  develop¬ 
ing  weapons  and  advanced  weapons  concepts.  The  China  Lake  facility 
consists  of  the  Michelson  Laboratories,  the  Naval  Air  Facility,  and 
extensive  ground,  track,  and  aircraft  test  ra.-.g“?.  The  majority  of 
ranges  are  used  for  development  testing.  However,  some  OT&E  is 
conducted  here  by  VX-5  and  fleet  units.  Of  special  applicability  to 
OTLE  are  the  COSO  and  Echo  Ranges.  The  COSO  Range  is  an 
instrumented  facility  that  closely  simulates  a  combat  environment 
and  target  conditions  (although  no  live  ordnance  is  dropped).  The 
Echo  Range  provides  an  extensive  electronic  warfare  environment. 

Comment# 


(1)  Funding:  The  range  at  China  Lake  is  industrially  funded. 
They  feel  that  this  makes  for  more  efficient  management,  although  it  is 
hard  to  compete  with  free  ranges  as  they  do  have  to  hire  and  fire 
according  to  workload. 

(•2)  Hostile  Environment:  The  Echo  Range  simulates  hostile 
environments  and  provides  an  excellent  range  to  test  the  effectiveness 
of  OECM  gear  as  well  as  the  tactics  to  be  used.  It  aiso  provides 
excellent  development  testing  capability.  Training  of  deploying  pilots 
in  the  best  use  of  their  OECM  gear  is  a  secondary  mission. 

The  COSO  Range  provides  enemy  vehicle  i,  radar,  etc., 
in  a  wooded  or  camouflaged  complex  to  test  the  ability  of  a  weapon 
system  and  pilot  to  acquire  and  attack  enemy  targets. 

(3)  Inter-Servicc  Use  of  Range;  The  various  ranges  in  the 
China  Lake  complex  are  used  for  development  tests  and  evaluation. 

The  other  Services  are  welcome  and  do  use  the  various  ranges  periodi¬ 
cally,  There  have  been  no  problems  establishing  priorities.  However, 
other  users  have  to  be  able  to  afford  the  cost  of  using  an  industrially- 
funded  facility. 
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(4)  Need  for  new  range  facilities  and  methods  of  controlling 
their  use:  Tire  Naval  Weapons  Center  sees  a  need  to  keep  updating 
facilities  to  provide  more  realistic  combat  conditions.  Control  of  use 
should  be  left  at  a  local  level.  Tnere  are  no  inter-Service  problems 
at  that  level. 

(5)  Scheduling:  There  is  a  scheduling  conference  with 
users  every  Thursday.  Bi.»ed  on  forecast  workload  and  instrumenta¬ 
tion  required,  the  following  week’s  schedule  is  planned.  Priorities 
are  seldom  a  problem  and  are  solved  by  local  arbitration.  The 
schedule  is  very  flexible.  They  are  usually  able  to  slip  in  another 
project  if  the  one  scheduled  has  to  cancel.  The  policy  is  to  accommodate 
all  customers  if  they  have  the  money. 

(6)  A  small  high-level  testing  agency  might  be  of  value  in 
conducting  large  scale  exercises  to  define  information  needed,  tabulate 
and  evaluate,  results,  and  most  important,  take  necessary  action  to 
correct  equipment  deficiencies. 

(7)  There  is  a  lack  of  information  concerning  what  some  of 
our  complete  systems  can  do  under  combat  conditions.  This  needs 
high-level  emphasis  and  representation,  perhaps  at  the  JOS  level 
where  the  Services  operate  together.  Any  higher-level  control  tends 
to  be  less  and  less  productive. 

PACIFIC  MISSILE  RAN~-E.  POINT  MUGU,  CALIFORNIA  (NATIONAL  RANGE) 


The  Pacific  Missile  Range  is  a  designated  National  Range  with  the 
U.S.  Navy  as  Executive  A,  ent.  The  range  headquarters  is  located  at 
Point  Mugu,  California  (approximately  fifty  miles  north  of  Los  Angeles, 
California).  The  ranges  tLcti  selves  consist  of  extensive  airspace  off 
the  rnaat  of  California  -  inriiul  *>v  some  portions  of  the  Channel  Islands  - 
and  an  underwater  range  in  the  'awaiian  Islands.  The  offshore  air¬ 
space  is  crossed  in  several  plac>  •  '  v  civil  airways;  permission  to  use 
this  airspace  is  granted  to  civilian  •.  e  on  an  "as  available"  basis  by  the 
Pacific  Missile  Range, 

Range  clearance  for  water  areas  is  maintained  by  dedicated 
resources  (helicopters,  etc.  )  that  patrol  the  areas  to  be  used  and  warn 
off  any  vessels  that  may  be  In  a  potential  impact  ate. 
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Comment* 


The  range  facilities  are  outstanding.  Ranges  are  used  for 
development  testing,  OTfcE  testing  and  training.  Scheduling  is  carried 
out  in  three  phases: 

(1)  Long  term  -  consists  of  planning  and  making  arrangements 
for  procurement  of  instrumentation  peculiar  to  a  planned  project.  Such 
instrumentation  is  funded  by  the  Project  Manager. 

(2)  Monthly  and  weekly  conferences  with  users  for  short 
range  planning. 

(3)  Daily  rescheduling  insofar  as  possible  to  substitute  for 
cancelled  events.  Priorities  are  determined  by  the  Pacific  Missile 
Range  in  arbitration  with  users;  there  have  been  no  problems  to  date. 

Air  Force  users  account  for  approximately  twenty-eight  per  cent, 
mostly  missile  firing;  NASA,  contractors,  and  various  other  agencies 
approximately  twelve  per  cent;  the  remainder  is  used  by  the  Navy. 

It  was  stressed  that  scheduling  has  to  be  done  locally  for  efficient 
use  of  the  range.  Funding  is  provided  by  the  Naval  Air  Systems 
Comma.'d  for  level-of-effort  plus  a  small  amount  for  range  improve¬ 
ments. 

Problems 

(1)  Encroachment  by  new  Los  Angeles  Airport  and  overseas 
airways  traffic. 

(2)  Oil  drilling  and  exploration  rigs. 

(3)  Proximity  of  tests  to  Russian  electronic  information 
gathering  ships  off  the  coast. 

I’SA  F  TACTICAL  FIGHTER  WEAPONS  CENTER,  NELLIS  AIR  FORCE 
BASF,  LAS  VEGAS,  NEVADA 

The  USAF  Tactical  Fighter  Weapons  Center  at  Nellis  Air  Force 
Base,  Nevada,  has  the  dual  mission  of  training  tactical  fighter  pilots 
(Including  the  development  of  tactics  and  doctrine)  and  OTfcE.  The 
Nellis  Air  Force  Base  ranges  lie  to  the  north  and  cast.  Tiicrc  arc 


significant  amounts  of  airspace  and  grounc  available  for  a  variety  of 
capabilities.  Because  Nellis  Air  Force  Base  has  in  the  past  been 
primarily  involved  with  training  missions,  the  ranges  were  so 
developed  to  fulfill  training  requirements.  At  present,  the  Tactical 
Fighter  Weapons  Center  has  one  rar>;'e,  Range  3,  dedicated  to 
formal  OT&E. 

Comments 

(1)  Range  facilities  for  OT&E  are  quite  primitive.  However, 
they  do  have  a  program  for  improvement  that  could  provide  a  good 
capability  if  sufficient  funds  are  provided. 

(2)  The  Tactical  Fighter  Weapons  Center  philosophy  is  that 
OT&E  must  be  responsive  to  urgent  needs,  and  therefore  ranges  must 
be  equally  responsive.  The  capability  to  use  ranges  for  both  testing 
and  training  is  necessary  for  their  mission.  They  do  not  think  an  OSD 
central  act’vity  would  enhance  responsiveness  to  the  users'  heeds  - 
for  development,  yes,  but  not  for  OT&E. 

(3)  Tactical  Fighter  Weapons  Center  personnel  appreciate 
the  capabilities  of  nearby  ranges  such  as  the  Naval  /capons  Center 
and  the  Pacific  Missile  Range.  However,  the  Nava'  Weapons  Center 
is  costly,  and  use  of  the  Pacific  Missile  Range  presents  problems 

in  overflight  and  staging  that  increase  costs  and  limit  responsiveness. 
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OFFICIAL  Pi'BLICATItWS  OF  A  DIRECTIVE  MATURE 

Office  of  the  Secretary  of  Detente 

Department  of  Defense  Directive  5129.1,  "Director  of  Defense 
Research  ar.d  Engineering,"  Office  of  the  Secretary  of 
Defense,  Washington,  D.  C.,  10  Feb  1959 

Describes  the  mission,  organization  and  operating 
procedures  of  DDR&E. 


Department  of  Defense  Directive  5158.1,  "Organization  of  the 
Joint  Chiefs  of  Staff  and  Relationships  with  the  Office 
of  the  Secretary  of  Defense,”  Office  of  the  Secretary  of 
Defense..  Washington.  D.  C.,  31  Dec.  1958 

Describes  the  mission,  organization  and  operating 
procedures  of  the  JCS  and  Joint  Staff. 


Department  of  Defense  Instruction  5129.37,  "Weapons  Systems 
Evaluation  Group,"  Office  of  the  Secretary  of  Defense, 
Director  of  Defense  Research  and  Engineering,  Washington, 
D.  C.,  23  August  1962 

Describes  the  fBisslon,  organization  and  operating 
procedures  of  WSEG  and  its  relationship  to  the 
JCS  and  DDR&E. 


Joint  Chiefs  of  Staff 

The  Joint  Chiefs  of  Staff  Memorandum  for  the  Secretary  of 

Defense,  JCSM-350-68,  Subject:  "Consideration  of  Requirement 
for  a  Joint  Test  and  Evaluation  Agency  (U) ,  31  May  1968 
(SECRET) 


JCS  reply  to  thj  Deputy  Secretary  of  Defense  proposal 
for  an  OT&E  agency  on  the  Joint  Staff. 


Department  of  the  Argf 


Army  Regulation  70*10,  "Research  &  Development:  Teat  & 

Evaluation  During  Research  and  Development  of  Materiel, " 
Headquarters,  Department  of  the  Army,  Washington,  D.  C., 
26  Dec.  I960 

The  current  policies  and  procedures  for  Army  teat 
and  evaluation  during  research  and  development  of 
materiel.  A  revision  to  this  AR  is  under  prepare* 
tion  to  inclement  new  procedures  for  OT&E. 


Army  Regulation  71-3,  "Force  Development:  User  Field  Tests, 
Experiments  6  Evaluation,"  Headquarters,  Department  of  the 
Army,  Washington,  D.  C.,  19  March  1968 

Outlines  the  objectives,  policies,  responsibilities 
n  'J  procedures  for  conduct  of  user  field  tests, 
experiments  and  evaluations.  These  tests  include 
troop  tests,  confirmatory  tests,  field  experiments, 
field  evaluations  and  combat  evaluations. 


Army  Regulation  71-6,  "Force  Development:  Type  Classification/ 
Reclassification  of  Army  Materiel"  (Effective  l  Jan  1970', 
Headquarters,  Department  of  the  Army,  Washington,  D.  C., 

20  Nov.  1969 

Describes  the  process  for  type  classification  and 
reclassification  of  items  of  Army  materiel  and 
establishes  overall  policies  and  procedures  for 
type  classification  or  tetlassification  actions. 

It  is  current  fjan,  70)  and  pro”'des  a  good  under¬ 
standing  of  how  the  Army  classifies  equipment  for 
its  use. 


Army  Regulation  705-5,  "Research  &  Development  of  Materiel: 
Army  Research  6  Development,"  Headquarters,  Department  of 
the  Army,  Washington,  D.  C.,  9  Aoril  1968;  Change  1, 

8  Dec.  1969 

Establishes  responsibilities,  policy  snd  procedures 
for  conducting  research  mid  development  in  the  Army. 
Provides  a  good  description  of  the  weapons  system 
acquisition  process. 
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Depart, sent  of  the  Army  Directive,  Subject:  "Test  Dir  etive 
for  Project  Mobile  Amy  Sensor  System  Test,  Evaluation  and 
Review  (MASSTER),"  Department  of  tne  Army,  Office  of  Chief 
o*  Staff,  29  Dec.  1969,  with  inclosure: 

Inclosure  l  -  "Project  MASSTER  Test  Schedule  -  FY  1970" 

Establishes  objectives  and  concept  of  operation 
of  Project  MASSTER. 


Department  of  the  Army  Letter  ACDA(M)  (13  Oct.  69)  CSSTANSO-M, 
Subject:  "Charter,  Project  Director,  Project  Mobile  Army 
Sensor  Systems  Test,  Evaluation,  and  Review  (MASSTER)," 
Department  of  the  Army,  Office  of  Adjutant  General, 
Washington,  D.  C. ,  27  Oct.  1969,  with  inclosures: 

Tnclosure  1  -  Department  of  the  Army  Letter  ACSD-C(M) 

(10  Sep  69)  CSSTANSO-C ,  Subject:  "Establishment  of  the 
US  Army  Project  Mobile  Army  Sensor  Systems  Test, 

Evaluation  and  Review  (MASSTER),"  Department  of  the  Army, 
Office  of  Adjutant  General,  Washington,  D.  C.,  13  Sep.  1969 
Inclosure  2  -  Charter  of  Project  Director  Mobile  Array 
Sensor  Systems  Test,  Evaluation  end  Review  (MASSTER),  1969 

Establishes  mission  for  Project  MASSTER 


Department  of  the  Army,  Office  of  the  Chief  of  Staff 

Memorandum  for  Record,  CSAVCS-F-ASG;  Subject;  "AVC  of  SA's 
30  Oct.  69  Meeting;  Operational  Test  6  Evaluation  &  Role 
of  Center  Commanders,"  31  Oct.  1969 

Providas  a  summary  of  a  high-level  me-'ting  chaired 
by  AVC/SA  on  the  subject  of  OT-iE  and  the  role  of  the 
Center  Commanders,  Discusses  future  steps  which  will 
be  taken  to  Improve  OT&E  and  to  obtain  maximum  bene¬ 
fits  from  timely  evaluation  of  test  results  at  all 
decision  levels. 


Department  of  Army  Pamphlet  No.  11-25,  "Life  Cycle  Mnagement 
Model  for  Army  Systems,"  Headquarters,  Department  of  the 
Army,  Washington,  D,  C.,  11  Oct.  1968 

This  pamphlet  supplements  AR  11-25,  "The  Management 
Process  for  the  Development  of  Army  Systems."  It 
describes  in  great  detail  the  steps  in  the  life  cycle 
process  by  which  Army  systems  are  developed, 
fielded  and  modified. 
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U.  S.  Army  Combat  Developments  Command  Regulation  71-7, 

"Combat  Developaents  Field  Experimentation,”  Headquarters, 
V.  S.  Army  Cuabat  Developments  Coanand,  Ft.  Belvoir,  Va., 
6  Jan.  1973 

Prescribes  policies,  responsibilities  for  the  conduct 
of  field  experiments  by  the  USACDC  Experimentation 
Coanand  (CDEC) . 


Department  of  the  Wavy 

COMOPTEVFOR  Instruction  3930. IE,  "Project  Instructions," 

3  Volumes,  Department  of  the  Navy,  Operational  Test  and 
Evaluation  Force,  Norfolk,  Va.,  6  March  1968  with  Changes 

Guidance  of  subordinate  commands  and  personnel 
concerned  with  the  prosecution  of  projects  for 
Commander,  Operational  Test  and  Evaluation  Force. 
Delineates  in  detail  the  methods  of  testing, 
test  plans  and  reporting  for  those  conducting 
tests. 


NAVAIR  Instruction  6200.12,  "Responsibilities  and  Procedures  for 
Release  for  Production  of  Systems,  Weapons  and  Equipment," 
Department  of  the  Navy,  Naval  Systems  Command,  Washington, 

D.  C.,  12  April  1968 

Naval  Air  Systems  Coimand  policies,  responsibilities, 
and  procedures  governing  the  release  for  production 
of  systems,  weapons  ar.d  equipment.  Delineates 
Naval  Aviation  policy  for  the  complicated  problem 
of  "fly  before  buy." 


OPN.iVINST  3930. 8A,  "OPNAV  Internal  Management  Procedures  for 
RDT6E  Projects  Assigned  to  the  Operating  Forces^" 

Department  of  the  Navy,'  Office  of  Chief  of  Naval  Operations, 
Washington,  D.  C.,  31  July  1968 

Cuidance  for  the  internal  management  and  initiation 
within  OPNAV  of  RDT6F.  projects  which  are  to  be 
assigned  to  the  Operating  Forces  for  operational 
test  and  evaluation.  Delineates  management  of 
C0M0PTF.VF0R  within  OPNAV  for  0T6.E. 
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OPNAVINS?  3960. ID,  "Prosecution  by  the  Operating  Forces  of 
CNO  Assigned  RDT5.E  Projects,"  Department  of  the  Navy, 
Office  of  Chief  of  Naval  Operations,  Washington,  D.  C., 
4  Dec.  1967 

Policy  and  guidance  for  the  prosecution  of  CNO 
assigned  RDT&E  projects  by  the  operating  forces, 
including  functions  and  r-...  >unsibilities  of 
RDT&E  activities  involved  .1  the  participation  of 
the  operating  forces.  Delineates  the  procedures 
whereby  the  operating  forces  support  the  RDT&E 
cycle. 


Department  of  the  Air  Force 


Air  Force  Regulation  No.  55-31,  Final  Draft,  "Operations: 
Operational  Test  and  Evaluation,"  Department  of  Air  Force, 
Washington,  D.  C. ,  1969 

States  the  objectives,  policies  and  responsibilities 
for  t’.  S.  Air  Force  Operational  Test  &  Evaluation 
activities,  including  the  development  of  employment 
concepts,  tactics,  and  techniques,  for  systems, 
subsystems,  and  equipment.  It  applies  to  all  Air 
Fctce  organizations  and  activities. 


Air  Force  Regulation  Ro.  80-14,  "Research  and  Development: 

Test  and  Evaluation  of  Systems,  Subsystems,  and  Equipment," 
Department  of  Air  Force,  Washington,  D.  C.,  24  Feb.  1967 

States  the  objectiv's,  oollcies  and  responsibilities 
for  U.  S.  Air  Fotce  Jt  and  evaluation  activities 
which  support  Air  Force  research  and  development, 
acquisition  of  operational  and  support  systems, 
subsystems,  and  equipment;  technical  and 
engineering  service  programs  and  projects.  It 
applies  to  all  Air  Force  organizations  and 
activities. 


Air  Force  Systems  Command  Pamphlet,  AFSCP  80-5,.  "Research  and 
Development:  A  Cuide  for  Design,  Conduct,  and  Analysis  of 
Air  Force  Tests,"  headquarters  AFSC,  Washington,  D.  C., 

1  April  1966 

For  I),  S.  Air  Force  personnel  involved  in  planning, 
designing,  directing  and  analyzing  the  results  of 
tests  and  evaluations  of  Air  Force  systems,  subsystems, 
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components,  and  equipment  items.  In  essence,  it  intro¬ 
duces  any  reader  not  specifically  trained  in  testing  to 
the  spectrum  of  considerations  he  will  encounter  -  from 
sit. pie  statistical  theory  to  Instrumentation,  data  col¬ 
lection  and  processing,  the  mechanics  of  good  test 
management,  and  the  presentation  of  results. 


STUDIES.  REPORTS  AND  OTHER  REFERENCES 


"Aircraft  Weapons  Systems  Test  and  Evaluation  Ad  Hoc  Committee 
Report,"  submitted  to  CDR  Navil  Air  Systems  Cotmand  by  Naval 
Air  Test  Center,  U.S.  Naval  Air  Station,  Patuxent  River,  iid., 
14  June  1967  (Does  not  include  Appendix  X) 

A  review  of  test  and  evaluation  in  Naval  Aviation.  Pro¬ 
vides  an  in-depth  study  of- testing  in  Naval  Aviation  and 
delineates  the  problems  of  management  control. 


"Analysis  of  Test  and  Selection  Procedures  for  Small  Arms-Lubricants," 
Report  No.  RACTC-TR-61,  prepared  for  Office  of  Advanced  Engineer¬ 
ing,  Advanced  Research  Projects  Agency,  DoD,  by  .-tACTC,  Battelle 
Memorial  Insticule-Columhus  Laboratories  and  A.  D.  Little,  Inc., 
Contract  No.  F33657-67-C-C810,  10  June  1968  (FOR  OFFICIAL  USE  ONLY) 

Reviews  the  history  of  the  M-16  rifle  and  of  the  -'arious 
small  arms  lubricants.  Results  of  combat  experimentation 
with  other  than  the  specified  lubricant  are  reported. 

Small  arms  development  cycle  is  described.  There  is  an 
extensive  bibliography  on  the  M-16. 


"Army's  'Unbelievable'  Mismanagement  Caused  M-16  Rifle  Problems, 
Armed  Services  Subcommittee  Found,"  Press  Release  by  U.S.  HouFe 
of  Representatives,  Committee  on  Armed  Services,  90th  Cong., 

19  Oct  1967 

Summarizes  findings  of  the  House  Armed  Services 
Subcommittee  investigating  the  M-16  malfunctions 
in  Southeast  Asia. 


"Dnl  Bulletin;  The  VX-5  Newsletter,"  Air  Test  and  Evaluation 
Squadron  FIVE,  U.S.  Naval  Air  Facility,  China  Lake,  Calif. 

This  newsletter  ia  distributed  to  Navy  operational 
units  and  contains  up-to-date  information  derived 
from  VX-5's  tests  and  evaluations  which  would  assist 
those  units  In  training  for  and  performing  combat 
missions. 
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"Bibliography  on  JfF-2"  prepared  by  Director  of  Operations, 
Headquarters  USAF,  Feb.  1969 

An  annotated  bibliography  for  internal  use  includes 
reports  on  formation  >f  JTF-2,  terms  of  reference, 
programs  and  funding;  exchange  of  information  with 
the  UK,  tests  and  related  data.  Copy  on  file  in 
AFXOW. 


"Concept  for  a  Program  of  Strategic  Low-Altitude  Penetration 
Tests  (U),"  Supplement  1  to  WSEG  Report  74,  Weapons 
Systems  Evaluation  Croup,  DoD,  June  1964  (SECRET) 

Provides  a  concept  and  integrated  test  program 
for  weapons  systems  tests. 


"Concepts  for  a  Program  of  Tactical  Low-Altitude  Penetration 
Tests  (U),"  WSEG  Report  74,  Weapons  Systems  Evaluation 
Croup,  DoD,  May  1964  (SECRET) 

Provides  a  concept  and  integrated  test  program 
for  weapons  systems  tests. 

"Conduct  an  Operational  Evaluation  of  the  Helmet  Sight  In  An 
F-4  Aircraft  As  An  Acquisition  Aid  During  Air  Combat 
Maneuvering  (U),"  First  Partial  deport  on  Project  0/V63 
Task  II  Annex  A,  Department  of  the  Navy,  Operational  Test 
and  Evaluation  Force,  Norfolk,  Va. ,  6  Oct  1969 
(CONFIDENTIAL) 

Report  on  a  recent  test  conducted  by  the  Navy's 
Operational  Test  and  Evaluation  Force,  Cited  as 
representative  of  the  activities  encompassed  by 
OPTEVFTR's  mission. 


"Department  of  the  Navy  KDT&E  Management  Guide,"  NAVSO  P-2457 
(Rev.  7-69),  Dept,  of  the  Navy,  Office  of  Asst.:  Secretary 
of  the  Navy  (RAD),  Washington,  D.  C.,  I  July  1969 

Provides  a  sunmary  overview  of  Dept,  of  Navy 
RDT6E  "machinery,"  and  la  a  handy  source  of  general 
Information  concerning  RDT&K  organization, 
procedures  and  references  to  official  sources. 
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"9*fe.i£<!  Scii  'ce  B.  *rd  Task  Fc'cs  on  RU>  Management," 
hf i  ininary  Draft,  of  Final  Report,  Office  of  Director 
Defetse  Keseoxch  and  Eng  it  je-liu;.  nag  196* 

Addi«-ss'-s  the  acquisition  ranagtiaenc  practices  at 
the  200  *lth  emphasis  on  r  *  iW£>  aspects.  Major 
focal  >cinr*  of  discussion  were  prototypes, 
requirement: ,  content  formulation  in  relation  to 
dt /elntwoent  needs  n-.‘  contracting  flexibility. 

Also  addressed  was  iic'o  maragenent  ir  of.e  weapon 
system  acquit; .t ion  process,  development  of  a 
management  model,  budgeting  c  J  costing. 


"Detemi-iation  of  USAF  Testing  Policies  and  Concepts  Whicr.  Best 
help  to  Achieve  Operational l\  Effective  Weapons  ai..i 
Equipnett,”  Thesis  by  Col.  C.  W.  I.utz,  USAF,  submitted  to 
Georg.  'cashing  ton  University,  1959 

A  thesis-form  discussion  of  the  problem  written 
by  Colonel  T.utt  as  a  part  of  his  requirements  for 
aa  advanced  decree  at  George  Washington  University. 
Although  written  in  1959,  much  of  Colonel  Lutz's 
material  and  ratl-nale  it  pertinent  to  present-day 
weapon  system  evaluation  pi  ilosophy.  Tne  basic 
requirement  for  test  of  Ai.  Fore;  equipment,  a  survey 
of  test  objectives,  concepts,  and  policies,  and 
conclusions  e"d  recommendations  for  optimum  teat 
management  doctrine  are  offered.  Support  for  hit 
findings  is  given  in  an  analysis  of  a  sampling  of 
opinion  of  test-experienced  people  on  the  staff  o. 
Headquarters,  USAF.  Comparison  of  testing  methodolo¬ 
gies  of  various  foreign  military  Aeronautical 
organizations  es  well  as  U.  S.  Civil  Aviation  authority 
it  offered. 


"Documentation  Alda  for  CINUSTRIKK  Test  and  Evaluation  Plan," 
prepared  by  Splndletop  Research,  toe.  for  Headquarters, 
USAF  Tactical  Air  Warferc  Center,  Eglin  Air  Force  Base, 
Fla.,  Conti  net  No.  AK  08(015)-«129,  I  June  1964  (FOR 
OFFICIAL  USE  ONLY! 

A  compilation  of  references  on  FIRE  SUPPORT, 
Rtl-ONhAlSFANCE,  TACTICAL  AIR  MOVEMENT ,  STRATEGIC 
AIR  NOVKM  INT ,  COMMAND  AND  CONTROL,  nnd  AIRCRAFT 
VUt.NrRAHILl'lY/SUftVfVAlirLirf.  The  purpose  was  to 
provide  basic  reference  information  tor  the  newly- 
otganlzed  Air  Force  Tactical  Air  Warfare  Center. 
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”DoD  Operational  Test  &  Evaluation  Report,"  unpublished  working 
paper  by  Or.  U'ic.  Pettijohn.,  no  date  (TOP  SECRET) 


Working  paper  includes: 

1.  Case  Studies  -  M151  iT.  Truck  (FOR  OFFICIAL  USE  ONLY) 

Short  Range  Cro;:nd  Surveillance  Radars  (CONFIDENTIAL) 
Tactical  Surtace -Co-Air  Missiles  (SECRET);  Strategic 
Missile  Testing  (TOP  SECRET) 

2.  "Section  VI:  Conclusions,"  fr.^n  a  Report  of  JTF-2 
Experiment  4.1  <'In-Ho-ise  Comments) ,  no  date 
(UNCLASSIFIED) 

3.  "Field  F.xperimentation  and  Small  Arms  Evaluation:  A 
Case  in  Point  Regarding  the  Future  of  Military  System 
Analysis,"  by  J.  A.  Stuckfisch,  unpublished  paper  which 
is  a  revision  and  expansion  of  a  paper  presented  to 
Military  Applications  Section  of  the  31st  Meeting  of 

CRSA  in  New  York,  2  June  19('7  (UNCLASSIFIED) 

4.  "Operational  Test  &  Evaluation  in  the  Services,"  by 
W.  Pacti John.  (Source:  "Study  of  Army  Test  and 
Evaluation,"  Hay  1966  -  a  comparison  of  the  three 
Services.  (FOR  OFFICIAL  USE  ONLY l 

5.  "The  Arcy  s  V'eapon  System  Studies,  Test  &  Evaluation  (U)," 
author  unknown,  paper  dated  15  June  1967  (FOR  OFFICIAL 
USE  ONLY) 

6.  "Inctiuroentation,"  by  W.  Pettijohn  (FOR  OFFICIAL  USE 
ONLY) 

7.  Tims  Schedule  FY  69-73  for  Various  Systems  from 
Production  Time  Through  Time  in  Procurement  Channels 
(UNCLASSIFIED) 

f>.  "7FKET  Instrumentation  System,"  .’glir.  Air  Force  Base,  Fla., 
1  Ju.y  1967,  17  p.  (UNCLASSIFIED) 

9.  "JTF-2  Instrumentation, "  excerpt  from  another  unidentified 
report,  5  p.  (UNCLASSIFIED) 

10.  Conclusions  &  Recormendations  I.  tJ.  Pettijohn  (FOR 
OFFICIAL  USE  ONLY) 

Tnese  papers  were  assembled  by  Dr.  Pettijohn  while 
ha  was  Special  Assistant  for  Operations  Test  and 
Evaluation,  OASD  (Systems  Analysis).  They  were 
assembled  in  1968,  for  the  most  part. 


"Evaluation  Methods  for  CINCSTRIKE  Test  and  Evaluation  Plan," 
prvprred  by  Splndtctop  Research,  Inc.  for  Headquarters, 
USAF  Tactical  Air  Warfare  Center,  Eglin  Air  Force  Base, 
Fla.,  Contract  No.  AF  Od(635) -4129,  April  1964  (FOR 
OFFICIAL  USE  ONLY) 
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Describes  testing,  war  genes,  and  analytical 
studies  by  which  the  Air  Force's  Tactical  Air 
Warfare  Center  proposed  to  support  C INC STRIKE 
in  conducting  rOLD  FIRE  I. 


"Hearings  Before  the  Special  Subcommittee  on  the  M-16  Rifle 
Program  of  the  Committe?  on  Armed  Services,"  No.  19, 

U.  S.  House  of  Representatives,  90th  Cong.,  1st  Sess., 

May  15,  16,  31,  June  21,  July  25,  26,  27,  Aug.  8,  9,  22,  1967 

Summarizes  months  of  study  including  field 
investigatiors  in  South  Vietnam  and  extensive 
hearings  in  Washington  on  M-16  malfunctions. 

It  includes  testimony  from  the  inventor, 
manufacturers  representatives,  cartridge 
propellant  representatives.  Army  program 
managers  and  comoat  personnel. 


"Integrated  Management  of  Test  and  Test  Support  Capabilities," 
Thesis  by  Charles  >!.  Ham,  submitted  to  Graduate  School  of 
Florida  State  Univers'ty,  April  1963 

Examines  the  feasibility  snd  desirability  of 
alternative  methods  of  achieving  the  integrated 
management  of  the  test  and  test  support 
capabilities  of  a  major  U.  S.  Air  Force  R4D  test 
organization.  At  the  time  the  research  was  done, 
the  author  had  more  than  20  years'  experience  in 
administering  OT&E  projects  and  facilities. 


"JCS  JTF-2  Final  Report  (U),"  Joint  Task  Force,  Sandla  Base, 
New  Mexico,  December  1968  (SECRET) 

Sunmarlzea  history  and  testing  program  of  JTF-2. 


"Journal  of  Defense  Research,  Series  B:  Tactical  Warfare  (U)," 
Vol,  IB  No.  3,  Fall  1969,  published  by  Institute  for  Defense 
Analysis  for  ARPA,  DoD,  Washington,  D.  C.  (SECRET) 

This  Issue  Is  devoted  exclusively  to  a  series  of 
papers  dealing  with  remote  -  sensors  technology 
applicable  to  counter  Infiltration  problems  and 
tactical  battlefield  problems.  This  overnll 
program  Is  managed  by  the  Defense  Communications 
Planning  Croup  (DCPG) ,  and  the  genesis  of  the 
concept  and  the  origins  of  DCPG  are  recorded. 
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Letter  to  M.  E.  Mars ton,  OSD  Blue  Ribbon  Defense  Panel, 

Wash.,  D.C.  from  RACIC  Battelle  Memorial  Institute, 

Columbus  Laboratories,  Reference:  R-3295,  18  Dec.  1969, 
with  enclosures: 

Enclosure  1  -"BibHoRraphy~Ml6  Rifle  Test  and  Evaluation*' 
Enclosure  2  -Letter  to  LtC  W.  J.  Lynch,  ARP4,  Wash.,  D.C. 
from  R.  W.  MacDonald,  Cooper-Macdona Id ,  Inc.,  Baltimore, 
Mi.,  10  Nov  1961,  with  enclosures: 

Enclosure  1  -  "History  of  AR-IS" 

Enclosure  2  -  Set  of  Pictures 

Letter  forwards  additional  bibliography  to  that  in 
RACIC-TR61,  "Analysis  of  Test  and  Selection 
Procedures  for  Small  Arms  Lubricants,"  and  a  letter 
from  Mr.  Macdonald,  President  of  Cooper-hacdona Id , 

Inc.  The  latter  sunssarizes  history  through  1961 
of  the  AR-15  (commercial  name  for  the  M-16) 
development.  It  summarizes  results  cf  Service 
comparative  tests  and  initial  world-wide  sales 
-efforts. 


"Operational  Reliability  Test  M-16A1  Rifle  System,"  WSEG  Report' 
124,  Weapons  Systems  Evaluation  Group,  DoD,  Feb  1968 
(FOR  OFFICIAL  USE  ONLY) 

Describes  an  operational  test  carried  on  by  the  U.S. 

Marine  Corps  under  WSEG  supervision  in  Panama,  tc 
measure  the  operational  reliability  of  the  M-16A1 
rifle  systems.  The  M-14  rifle  system  was  included,  in  the 
test  as  a  control.  Three  types  of  M-16A1  rifles  were 
used  and  two  types  of  cartridge  propellant.  328  USMC 
riflemen  each  fired  480  rounds/day  for  three  days  in 
each  of  four  differing  environments  using  447  rifles. 


"Papers  Concerning  TAUC  Vulnerabj lity/Survivability  Evaluations," 
prepared  by  Spindletop  Research,  Inc.  for  Headquarters,  USAF 
Tactical  Air  Warfare  Center,  Egiln  Air  Force  Base,  Fla., 
Contract  No.  AF  08(635) -4129,  June  1964  (FOR  OFFICIAL 
USE  ONLY) 

Doftls  with  the  problems  associated  with  slrcraft 
vulnerability  which  an  operational  testing 
organization  must  fawc  and  solve.  Also  contains 
a  bibliography  c.  vuincrnhility/iurvlvabil ity 
which  contains  274  references. 
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"Papers  Exemplifying  Field  Test  (Exercise);  D«t«  Collection 
Concepts,"  prepared  by  Spindletop  Research,  Inc.  for 
Headquarters,  I'SAF  Tactical  Air  Warfare  Center,  Eglin 
Air  Force  Base,  Fla.,  Contract  No.  AF  08(635)-4129, 

May  1964 

A  compilation  of  various  papers  prepared  for 
the  Commander,  Tactical  Air  Warfare  Center,  TAC, 
during  the  period  (i964)  when  TAWC  was  involved 
in  preparing  for  the  Joint  Army-Air  Force  exercises 
which  were  designated  GOLD  FIRE  I  and  GOLD  FIRE  II. 
They  deal  mainly  with  methods  of  obtaining  quanti¬ 
tative  data  from  large-scale  field  exercises  and 
cover  such  activities  as  Fire  Support,  Reconnaissance, 
and  Logistical  Support. 


"A  Perspective:  Air  Force  Testing  and  Evaluation,"  by  Charles 
C.  Ham,  in  Tactical  Air  Warfare  Center  Quarterly  Report, 
Dec.  1969 

Represents  an  overview  of  the  current  Air  Force 
testing  concept  and  presents  a  rationale  for  a 
possible  re-structuring.  The  author  bases  his 
views  on  what  he  regards  as  redundancies  and 
unnecessary  complexities  in _ the  managing  and 
conduct  of  today's  test  and  evaluation  program 
in  the  Air  Force. 


"Planning  Within  the  Department  ot  Defense  (U),"  Working  Paper, 
Report  by  Military  Aircraft  Panel  of  President's  Science 
Advisory  Comnittee,  18  April  1969,  23  p.  (CONFIDENTIAL) 

On  11  Aug.  1969  this  report  was  forwarded  to  the 
Secretary  of  Defense  by  Dr.  DuBrldge,  the  President's 
Science  Advisor,  with  the  coramnt  that  Mr.  Laird 
might  wish  to  make  it  available  to  the  "Fitshugh 
Commission."  It  has  a  section  devoted  to  an 
assessment  of  military  operational  testing  and 
evaluation. 


"Project  Deseret  Project  Master  Plan  (PMP)(U),"  RCS;  AMC  PM-101, 
Project  Deseret  Teat  Center,  Ft,  Douglas,  Utah,  30  Sept.  1969, 
circa  75p.  (SECRET) 

Clves  history,  organisation  and  schedule  for 
Deseret  Test  Center. 
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"Project  MAS5TER  (y) ,"  Briefing  presented  to  DoD  Sensor  Aided 

Combat  Symposium,  at  National  Bureau  of  Standards,  Caitliersburg, 
Md.,  by  Maj.  Con.  John  Norton,  U.S.-  Army,  Deputy  Director, 

Proj  ;ct  MASSTER,  .Ian.  1970  (CONFIDENTIAL) 

Explains  mission  and  tentative  approach  to  testing  of 
Advanced  Sensors  in  a  combined  arms  environment.; 

"Project  MASSTER  Organization  Chart,"  effective  6  Jan  1970 


"A  Prototype  Strategy  for  Aircraft  Development  (U) Memorandum 
RM-5597-PR,  by  Robert  L.  P.rry,  RAND  Corporation,  April  1968 
(CONFIDENTIAL) 

An  examination  of  the  conditions  that  warrant  the  use 
of  a  prototype  strategy  in  the  development  of  military 
aircraft.  Examples  are  given,  based  on  recent  experi* 
once  both  in  the  U.S.,  and  abroad.  Attention  is  given 
to  the  management  approach  that  is  most  appropriate  to 
a  prototype  strategy. 


"Report  of  the  Panel  on  R&D  Management,"  1969  Summer  Study, 
Defense  Science  Board,  Office  of  the  Director,  Defense 
Researcli  &  Engineering,  Newport,  Rhode  Island,  6-18  July  1969 

Study  formulates  specific  actions  which  OSD  'could 
consider  taking  to  Implement  the  recomnendations  of  the 
QS3  Task  Force  on  R&D  Management.  Consists  of  a 
summary  with  recommendations  for  action  together  with 
discussion,  and  several  supporting  papers  on  the 
principal  Issues. 


"The  Role  of  Competition  in  Aeronautics,"  presented  by  G.  S.  Schalrer, 
The  Boeing  Co.,  at  The  Wilbur  and  Orville  Wright  Memorial  Lecture 
of  tl  t\oyaL  Aeronautical  Society,  London,  England,  5  Dec  1968 

Emphasizes  that  competition  is  a  very  important  factor 
in  achieving  aeronautical  advances  -  thac  competition 
results  in  the  customer  receiving  more  for  his  money. 

Competition  can  be  limited  to  paper  studies,  to  proto¬ 
types,  or  continued  throughout  production.  Paper 
competitions  assume  that  people  get  smart  by  studying. 

Prototype  competitions  add  the  smartness  learned  only 
by  doing. 
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"A  Study  of  A ray  Test  end  Evaluation  (SATE),"  Volume  II: 
Summary  and  Section  V,  Improvement  Actions,  Department 
of  the  Army,  Office  of  Chief  of  Staff,  Systems  Analysis 
Croup,  May  1966  (FOR  OFFICIAL  USE  ONLY) 

Volume  II  provides  a  summary  of  the  SATE  study 
effort  and  a  detailed  discussion  of  the  eight 
improvement  actions.  Provides  background  infor¬ 
mation  as  to  .he  alternatives  available  in  1966 
for  revised  Army  testing  organization  and  procedures. 


"Test  Capabilities  at  Yuma  Proving  Ground  (Revised  Edition)," 
Department  of  the  Army,  Yuma  Proving  Ground,  Yuma,  Arizona, 
October  1968 

Provides  detailed  description  of  the  extensive  test 
facilities  and  capabilities  at  Yuma  Proving  Ground, 
Arizona. 


"U.S.  Air  Force  Tactical  Air  Support  Requirements  Board  Comments 
on  Report  of  Army  Tactical  Mobility  Requirements  Board  (U)," 

4  Volumes,  Department  of  the  Air  Force,  The  Tactical  Air 
Support  Requirements  Board,  Washington,  D.C.,  14  August  19(2 
(SECRET) 

This  report  was  prepared  by  a  board  of  general  offirers 
chaired  by  (then)  Lt.  Gen.  Disouay.  It  is  a  critique 
of  the  Army's  "Howze  Board"  report  and  also  contains 
resulcs  of  Air  Force  stjdies  and  tests  relating  to  the 
Air  Force's  capabilities  to  contribute  to  Army  air 
mobility. 


"Weapons  Systems  Evaluation  Gtoup,"  Weapons  Systems  Evaluation 
Group,  Department  of  Defense,  no  date,  9p. 

An  information  brochure  for  new  WSEG  personnel  which 
summarizes  WSKC  origin  and  hlslrry,  relationships  to 
LDR6K  nnd  JCS,  contractor  iclatlonshlp  and  support, 
and  nature  of  operational  evaluations. 
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BRIEFINCS  ATTENDED  BY  THE  OPERATIONAL  TEST  &  EVALUATION  TASK 
CROL'P  OF  THE  BLUE  RIB30N  DEFENSE  PANEL  STAFF 

Armv  or  Amy  Affiliated 

'The  Role  of  the  United  States  A  ray  Materiel  Command  in  A  ray 
Operational  Test  &  Evaluation,”  presented  by  LTC  Curtis 
Aater.de,  Deputy  Chief  of  Test  &  Evaluation  Division, 
Headquarters,  Army  Hateriei  Command,  Washington,  D.  C. 

"The  Role  of  the  United  States  Combr  :  Development  Cootnand  in 

Aray  Operational  Test  &  Evaluation,”  presented  by  Mr.  Jaca 
Harris,  Plans  &  Programs  Division,  Combat  Development 
Command,  Ft.  Belvoir,  Va. 

"United  States  Army  Test  &  Evaluation  Command,"  presented  by 
Mr.  Benjamin  S.  Goodwin,  Special  Assistant  to  the 
Conscanding  General,  Test  &  Evaluation  Command,  Aberdeen 
Proving  Ground,  Kd. 

"Yuma  Proving  Grounds  Test  Operations,"  presented  by  Mr.  Floyd 
Watts,  technical  Advisor,  Yuma  Proving  Grounds,  luma, 
Arizona. 

Navy  and  Marine  Corns  or  Navy  Affiliated 

"Naval  Operational  Test  and  Evaluation,"  presented  by  CAPT  James 
L.  Anderson,  Director,  Surface  Warfare  Division;  CAPT 
Richard  W.  Huxford,  Director,  Air  Warfare  Division;  and 
CDR  Rudolph  L.  Krause,  Head,  Fighter  Webpon  Section,  Air 
Warfare  Division,  Operational  Test  &  Evaluation  Force, 

U.  S.  Naval  Base,  Norfolk,  Va. 

"Naval  Weapons  Center,"  presented  by  CaPT  J.  K.  McConeghy,  Jr., 
Executive  Officer;  Mr.  H.  J.  Wilson,  Deputy  Technical 
Director;  Mr.  C.  J.  DIPol,  Associate  Dept.  Head 
(Operations);  Dr.  I.  E.  Highbcrg,  Head,  Systems  Development 
Dept.,  Naval  Weapons  Center,  China  Lake,  California. 

"Operational  Tost  &  Evaluation  in  the  United  States  Marine 

Corps,"  presented  ly  COL  2.  J.  Maloney,  Assistant  Deputy 
Chief  of  Staff  (RD&S) ,  Headouarters,  U.  S.  Marine  Corps, 
Washington,  D.  C. 

"Pacific  Missile  Range  Facilities,"  presented  by  CAPf  R,  T. 
Janiec,  Range  Operations  Officer,  Pacific  Missile  Range, 

Pt.  Mugu,  California. 

"Pacific  Missile  Ranga  Operation,"  presented  by  RADM  Howard  S. 
Moore,  Commander,  Pacific  Missile  Range,  and  CAPT  J.  0. 
Mooney,  Director,  Pacific  Missile  Range  Directorate, 

Pt,  Mugu,  California. 
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"Testing  Philosophy,"  presented  by  CAFT  George  Uatkina, 

Deputy  Director,  Naval  Air  Test  Center,  Patuxent  River,  Hd. 

"VX-A  Projects  and  Testing  Philosophy,"  presented  by  CAKT  Jaaes 
Foster,  Ccasaaiidlng  Officer,  VX-4-NAS,  Pt.  Hugo, 

California. 

"VX-5  Operations  and  Ceneral  Testing  Philosophy,"  presented  by 
CAPT  Charles  Fritt,  Commanding  Officer,  VX-5-NAS, 

China  Lake,  California. 

Air  force  or  Air  Force  Affiliated 

"HAVE  EDGE,"  presented  by  Hr.  Howard  A.,  Beck,  Chief,  Resources 

Branch,  Evaluation  and  Resources  Group,  Deputy  Director  for 
Operaticn.nl  Test  and  Evaluation,  DCS/Plans  A  Operations, 
Headquarters,  USAF,  Washington,  D.  C. 

"Strategic  Missile  Operational  Test  A  Evaluation,"  presented  by 
Lt.  Col.  Edgar  A.  Northrup,  Jr.,  Strategic  Division, 
Directorate  of  Operations,  Headquarters,  USAF,  Washington, 
D.  C. 

"United  States  Air  Force  Operational  Test  A  Evaluation," 
presented  by  Lt.  Col.  Malcolm  Ageev,  Deputy  Chief, 
Evaluation  A  Resources  Group,  Office  of  the  Deputy 
Director  for  Operational  Test  A  Evaluation,  DCS/Plans  A 
Operations,  Headquarters,  USAF,  Washington,  D.  C. 


Other 


"General  Motors  Proving  Ground  Operations,"  presented  by  Mr.  C. 
J.  Brady,  Manager,  Ceneral  Motors  Proving  Grounds,  Milford, 
Michigan. 

"The  Operational  Test  and  Evaluation  Stuoy  Conducted  by  MSEC 
Study  Group,"  presented  by  Dr.  Robert  Fox,  Director, 

Science  A  Technology  Division,  Institute  for  Defense 
Analyses,  Arlington,  Va. 

"Overall  View  of  General  Motors  Engineering  Concept,"  presented 
by  Mr.  L.  A.  Kintigh,  Vice-President,  General  Motors 
Engineering  Staff,  Ceneral  Motors  Technical  Center, 

Warren,  Michigan. 

"Research,  Development,  'cst  and  Evaluation  Policies  Within 

General  Motors,"  p  esented  by  Mr.,  K.  E,  Bruoker,  Manager, 
Engineering  Staff  Operations,  General  Motors  Engineering 
Staff,  General  Motors  Technical.  Center,  Warren,  Michigan. 

"The  Weapons  System*  Evaluation  Group,"  presented  by  Col. 
Clifford  Moore,  Jr.,  USAF,  Weapons  Systems  Evaluation 
Group,  Department  of  Defense,  Washington,  D.  C. 
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